Release Zulip Server 1.9.2.

This should hopefully be the last commit of this form; ultimately, my
hope is that we'll be able to refactor the semi-duplicated logic in
this file to avoid so much effort going into keeping this correct.

docs/conf.py

@@ -54,7 +54,7 @@ author = 'The Zulip Team'
 # The short X.Y version.
 version = '1.9'
 # The full version, including alpha/beta/rc tags.
-release = '1.9.0'
+release = '1.9.2'
 
 # This allows us to insert a warning that appears only on an unreleased
 # version, e.g. to say that something is likely to have changed.

docs/overview/changelog.md

@@ -7,6 +7,34 @@ All notable changes to the Zulip server are documented in this file.
 This section lists notable unreleased changes; it is generally updated
 in bursts.
 
+### 1.9.2 -- 2019-01-29
+
+This release migrates Zulip off a deprecated Google+ API (necessary
+for Google Authentication to continue working past March 7), and
+contains a few bug fixes for the installer and Slack import.  It has
+minimal changes for existing servers not using Google authentication.
+
+- Updated the Google Auth integration to stop using a deprecated and
+  soon-to-be-removed Google+ authentication API.
+- Improved installer error messages for common configuration problems.
+- Fixed several bugs in Slack, Gitter, and HipChat import tools.
+- Fixed a subtle bug in garbage-collection of the node_modules cache.
+- Optimized performance of Slack import for organizations with
+  thousands of users.
+
+### 1.9.1 -- 2018-11-30
+
+This release is primarily intended to improve the experience for new
+Zulip installations; it has minimal changes for existing servers.
+
+- Added support for getting multi-domain certificates with setup-certbot.
+- Improved various installer error messages and sections of the
+  installation documentation to help avoid for common mistakes.
+- The Google auth integration now always offers an account chooser.
+- Fixed buggy handling of avatars in Slack import.
+- Fixed nginx configuration for mobile API authentication to access uploads.
+- Updated translation data, including significant new Italian strings.
+
 ### 1.9.0 -- 2018-11-07
 
 **Highlights:**

docs/production/deployment.md

@@ -74,20 +74,117 @@ those providers, Zulip's full-text search will be unavailable.
 ## Putting the Zulip application behind a reverse proxy
 
 Zulip is designed to support being run behind a reverse proxy server.
-There are few things you need to be careful about when configuring a
-reverse proxy:
+This section contains notes on the configuration required with
+variable reverse proxy implementations.
+
+### Installer options
+
+If your Zulip server will not be on the public Internet, we recommend,
+installing with the `--self-signed-cert` option (rather than the
+`--certbot` option), since CertBot requires the server to be on the
+public Internet.
+
+#### Configuring Zulip to allow HTTP
+
+Depending on your environment, you may want the reverse proxy to talk
+to the Zulip server over HTTP; this can be secure when the Zulip
+server is not directly exposed to the public Internet.
+
+After installing the Zulip server as
+[described above](#installer-options), you can configure Zulip to talk
+HTTP as follows:
+
+1. Add the following block to `/etc/zulip/zulip.conf`:
+
+    ```
+    [application_server]
+    http_only = true
+    ```
+
+1. As root, run
+`/home/zulip/deployments/current/scripts/zulip-puppet-apply`.  This
+will convert Zulip's main `nginx` configuration file to allow HTTP
+instead of HTTPS.
+
+1. Finally, restart the Zulip server, using
+`/home/zulip/deployments/current/scripts/restart-server`.
+
+### nginx configuration
+
+You can look at our
+[nginx reverse proxy configuration][nginx-loadbalancer] to see an
+example of how to do this properly (the various include files are
+available via the `zulip::nginx` puppet module).  Or modify this example:
+
+```
+map $http_upgrade $connection_upgrade {
+        default upgrade;
+        ''      close;
+}
+server {
+        listen                  443 ssl;
+        server_name             zulip.example.net;
+
+        ssl                     on;
+        ssl_certificate         /path/to/fullchain-cert.pem;
+        ssl_certificate_key     /path/to/private-key.pem;
+
+        location / {
+                proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
+                proxy_set_header        Host $http_host;
+                proxy_set_header        Upgrade $http_upgrade;
+                proxy_set_header        Connection $connection_upgrade;
+                proxy_http_version      1.1;
+                proxy_buffering         off;
+                proxy_read_timeout      20m;
+                proxy_pass              https://zulip-upstream-host;
+        }
+}
+```
+
+Don't forget to update `server_name`, `ssl_certificate`,
+`ssl_certificate_key` and `proxy_pass` with propper values.
+
+[nginx-proxy-config]: https://github.com/zulip/zulip/blob/master/puppet/zulip/files/nginx/zulip-include-common/proxy
+[nginx-proxy-longpolling-config]: https://github.com/zulip/zulip/blob/master/puppet/zulip/files/nginx/zulip-include-common/proxy_longpolling
+[voyager.pp]: https://github.com/zulip/zulip/blob/master/puppet/zulip/manifests/voyager.pp
+[zulipchat-puppet]: https://github.com/zulip/zulip/tree/master/puppet/zulip_ops/manifests
+[nginx-loadbalancer]: https://github.com/zulip/zulip/blob/master/puppet/zulip_ops/files/nginx/sites-available/loadbalancer
+
+### HAProxy configuration
+
+If you want to use HAProxy with Zulip, this `backend` config is a good
+place to start.
+
+```
+backend zulip
+    mode http
+    balance leastconn
+    http-request set-header X-Client-IP %[src]
+    reqadd X-Forwarded-Proto:\ https
+    server zulip 10.10.10.10:80 check
+```
+
+Since this configuration uses the `http` mode, you will also need to
+[configure Zulip to allow HTTP](#configuring-zulip-to-allow-http) as
+described above.
+
+### Other proxies
+
+If you're using another reverse proxy implementation, there are few
+things you need to be careful about when configuring it:
 
 1. Configure your reverse proxy (or proxies) to correctly maintain the
 `X-Forwarded-For` HTTP header, which is supposed to contain the series
-of IP addresses the request was forwarded through.  This
-[nginx code snippet][nginx-proxy-config] will do the right thing, and
-you can verify your work by looking at `/var/log/zulip/server.log` and
-checking it has the actual IP addresses of clients, not the IP address
-of the proxy server.
+of IP addresses the request was forwarded through.  You can verify
+your work by looking at `/var/log/zulip/server.log` and checking it
+has the actual IP addresses of clients, not the IP address of the
+proxy server.
 
-2. Ensure your proxy doesn't interfere with Zulip's use of long-polling
-for real-time push from the server to your users' browsers.  This
-[nginx code snippet][nginx-proxy-longpolling-config] will do the right thing.
+2. Ensure your proxy doesn't interfere with Zulip's use of
+long-polling for real-time push from the server to your users'
+browsers.  This [nginx code snippet][nginx-proxy-longpolling-config]
+does this.
 
 The key configuration options are, for the `/json/events` and
 `/api/1/events` endpoints:
@@ -97,22 +194,11 @@ The key configuration options are, for the `/json/events` and
 * `proxy_buffering off`.  If you don't do this, your `nginx` proxy may
   return occasional 502 errors to clients using Zulip's events API.
 
-3. The other tricky failure mode with `nginx` reverse proxies is that
-they can load-balance between the IPv4 and IPv6 addresses for a given
-hostname.  This can result in mysterious errors that can be quite
-difficult to debug.  Be sure to declare your `upstreams` in a way that
-won't do load-balancing unexpectedly (e.g. pointing to a DNS name that
-you haven't configured with multiple IPs for your Zulip machine;
-sometimes this happens with IPv6 configuration).
-
-You can look at our
-[nginx reverse proxy configuration][nginx-loadbalancer] to see an
-example of how to do this properly (the various include files are
-available via the `zulip::nginx` puppet module).
-
-[nginx-proxy-config]: https://github.com/zulip/zulip/blob/master/puppet/zulip/files/nginx/zulip-include-common/proxy
-[nginx-proxy-longpolling-config]: https://github.com/zulip/zulip/blob/master/puppet/zulip/files/nginx/zulip-include-common/proxy_longpolling
-[voyager.pp]: https://github.com/zulip/zulip/blob/master/puppet/zulip/manifests/voyager.pp
-[zulipchat-puppet]: https://github.com/zulip/zulip/tree/master/puppet/zulip_ops/manifests
-[nginx-loadbalancer]: https://github.com/zulip/zulip/blob/master/puppet/zulip_ops/files/nginx/sites-available/loadbalancer
-
+3. The other tricky failure mode we've seen with `nginx` reverse
+proxies is that they can load-balance between the IPv4 and IPv6
+addresses for a given hostname.  This can result in mysterious errors
+that can be quite difficult to debug.  Be sure to declare your
+`upstreams` equivalent in a way that won't do load-balancing
+unexpectedly (e.g. pointing to a DNS name that you haven't configured
+with multiple IPs for your Zulip machine; sometimes this happens with
+IPv6 configuration).

docs/production/email.md

@@ -55,6 +55,30 @@ follows:
   providers
 * The password like `email_password = abcd1234` in `/etc/zulip/zulip-secrets.conf`.
 
+### Using system email
+
+If you'd like to send outgoing email using the local operating
+system's email delivery configuration (e.g. you have `postfix`
+configuration on the system that forwards email sent locally into your
+corporate email system), you will likely need to use something like
+these setting values:
+
+```
+EMAIL_HOST = 'localhost'
+EMAIL_PORT = 25
+EMAIL_USE_TLS = False
+EMAIL_HOST_USER = ""
+```
+
+We should emphasize that because modern spam filtering is very
+aggressive, you should make sure your downstream email system is
+configured to properly sign outgoing email sent by your Zulip server
+(or check your spam folder) when using this configuration.  See
+[documentation on using Django with a local postfix server][postfix-email]
+for additional advice.
+
+[postfix-email]: https://stackoverflow.com/questions/26333009/how-do-you-configure-django-to-send-mail-through-postfix
+
 ### Using Gmail for outgoing email
 
 We don't recommend using an inbox product like Gmail for outgoing

docs/production/export-and-import.md

@@ -55,6 +55,16 @@ archive of all the organization's uploaded files.
 
 ## Import into a new Zulip server
 
+The Zulip server you're importing into needs to be running the same
+version of Zulip as the server you exported from, so that the same
+formats are consistent.  For exports from zulipchat.com, usually this
+means you need to upgrade your Zulip server to the latest `master`
+branch, using [upgrade-zulip-from-git][upgrade-zulip-from-git].
+
+First [install a new Zulip server](../production/install.html),
+skipping "Step 3: Create a Zulip organization, and log in" (you'll
+create your Zulip organization via the data import tool instead).
+
 Log in to a shell on your Zulip server as the `zulip` user. Run the
 following commands, replacing the filename with the path to your data
 export tarball:

docs/production/mobile-push-notifications.md

@@ -25,9 +25,15 @@ follows:
    If you installed your Zulip server with a version older than 1.6,
    you'll need to add the line (it won't be there to uncomment).
 
-1. If you're running Zulip 1.8.1 or newer, you can run `manage.py
-   register_server` from `/home/zulip/deployments/current`.  This
-   command will print the registration data it would send to the
+1. If you're running Zulip 1.8.1 or newer, you can run the
+    registration command:
+    ```
+    # As root:
+    su zulip -c '/home/zulip/deployments/current/manage.py register_server'
+    # Or as the zulip user, you can skip the `su zulip -c`:
+    /home/zulip/deployments/current/manage.py register_server
+    ```
+   This command will print the registration data it would send to the
    mobile push notifications service, ask you to accept the terms of
    service, and if you accept, register your server.  Otherwise, see
    the [legacy signup instructions](#legacy-signup).

docs/production/multiple-organizations.md

@@ -50,6 +50,21 @@ For servers hosting a large number of organizations, like
 `ROOT_DOMAIN_LANDING_PAGE = True` in `/etc/zulip/settings.py` so that
 the homepage for the server is a copy of the Zulip homepage.
 
+### SSL Certificates
+
+You'll need to install an SSL certificate valid for all the
+(sub)domains you're using your Zulip server with.  You can get an SSL
+certificate covering several domains for free by using
+[our Certbot wrapper tool](../production/ssl-certificates.html#after-zulip-is-already-installed),
+though if you're going to host a large number of organizations, you
+may want to get a wildcard certificate.  You can also get a wildcard
+certificate for
+[free using Certbot](https://community.letsencrypt.org/t/getting-wildcard-certificates-with-certbot/56285),
+but because of the stricter security checks for acquiring a wildcard
+cert, it isn't possible for a generic script like `setup-certbot` to
+create it for you; you'll have to do some manual steps with your DNS
+provider.
+
 ### Other hostnames
 
 If you'd like to use hostnames that are not subdomains of each other,

docs/production/requirements.md

@@ -37,8 +37,12 @@ upgrading.
 
 If you're using Ubuntu, the
 [Ubuntu universe repository][ubuntu-repositories] must be
-[enabled][enable-universe], which is usually just `sudo
-add-apt-repository universe`.
+[enabled][enable-universe], which is usually just:
+
+```
+sudo add-apt-repository universe
+sudo apt update
+```
 
 [ubuntu-repositories]:
 https://help.ubuntu.com/community/Repositories/Ubuntu

docs/production/security-model.md

@@ -114,7 +114,7 @@ strength allowed is controlled by two settings in
     figure out whether a stream with that name exists, but cannot see any
     other details about the stream.
 
-  * See [Stream permissions](/help/stream-permissions) for more details.
+  * See [Stream permissions](https://zulipchat.com/help/stream-permissions) for more details.
 
 * Zulip supports editing the content and topics of messages that have
   already been sent. As a general philosophy, our policies provide
@@ -129,7 +129,7 @@ strength allowed is controlled by two settings in
     any time by that administrator.
 
   * See
-    [Configuring message editing and deletion](/help/configure-message-editing-and-deletion)
+    [Configuring message editing and deletion](https://zulipchat.com/help/configure-message-editing-and-deletion)
     for more details.
 
 ## Users and Bots
@@ -146,7 +146,7 @@ strength allowed is controlled by two settings in
   exceptions:
 
   * Administrators may get access to private messages via some types of
-    [data export](/help/export-your-organization).
+    [data export](https://zulipchat.com/help/export-your-organization).
 
   * Administrators can change the ownership of a bot. If a bot is subscribed
     to a private stream, then an administrator can indirectly get access to

docs/production/settings.md

@@ -16,7 +16,7 @@ administrator can do.  To change any of the following settings, edit
 the `/etc/zulip/settings.py` file on your Zulip server, and then
 restart the server with the following command:
 ```
-su zulip -c /home/zulip/deployments/current/scripts/restart-server
+su zulip -c '/home/zulip/deployments/current/scripts/restart-server'
 ```
 
 ## Specific settings

docs/production/ssl-certificates.md

@@ -72,6 +72,9 @@ The `--hostname` and `--email` options are required when using
 Zulip server machine to be reachable by that name from the public
 Internet.
 
+If you need to configure a multiple domain certificate, you can generate
+one as described in the section below after installing Zulip.
+
 [doc-install-script]: ../production/install.html#step-2-install-zulip
 
 ### After Zulip is already installed
@@ -80,11 +83,12 @@ To enable the Certbot automation on an already-installed Zulip
 server, run the following commands:
 ```
 sudo -s  # If not already root
-/home/zulip/deployments/current/scripts/setup/setup-certbot --hostname=HOSTNAME --email=EMAIL
+/home/zulip/deployments/current/scripts/setup/setup-certbot --email=EMAIL HOSTNAME [HOSTNAME2...]
 ```
 where HOSTNAME is the domain name users see in their browser when
 using the server (e.g., `zulip.example.com`), and EMAIL is a contact
-address for the server admins.
+address for the server admins. Additional hostnames can also be
+specified to issue a certificate for multiple domains.
 
 ### How it works
 

docs/production/upload-backends.md

@@ -35,25 +35,35 @@ created (e.g. `exampleinc-zulip-uploads`).
 1. Comment out the `LOCAL_UPLOADS_DIR` setting in
 `/etc/zulip/settings.py` (add a `#` at the start of the line).
 
+1. In some AWS regions, you need to explicitly
+    [configure boto](http://boto.cloudhackers.com/en/latest/boto_config_tut.html)
+    to use AWS's SIGv4 signature format (because AWS has stopped
+    supporting the older v3 format in those regions).  You can do this
+    by adding an `/etc/boto.cfg` containing the following:
+    ```
+    [s3]
+    use-sigv4 = True
+    ```
+
 1. You will need to configure `nginx` to direct requests for uploaded
-files to the Zulip server (which will then serve a redirect to the
-appropriate place in S3), rather than serving them directly.
+    files to the Zulip server (which will then serve a redirect to the
+    appropriate place in S3), rather than serving them directly.
 
-With Zulip 1.9.0 and newer, you can do this automatically with the
-following commands run as root:
+    With Zulip 1.9.0 and newer, you can do this automatically with the
+    following commands run as root:
 
-```
-crudini --set /etc/zulip/zulip.conf application_server no_serve_uploads true
-/home/zulip/deployments/current/scripts/zulip-puppet-apply
-```
+    ```
+    crudini --set /etc/zulip/zulip.conf application_server no_serve_uploads true
+    /home/zulip/deployments/current/scripts/zulip-puppet-apply
+    ```
 
-(The first line will update your `/etc/zulip/zulip.conf`).
+    (The first line will update your `/etc/zulip/zulip.conf`).
 
-With older Zulip, you need to edit
-`/etc/nginx/sites-available/zulip-enterprise` to comment out the
-`nginx` configuration block for `/user_avatars` and the `include
-/etc/nginx/zulip-include/uploads.route` line and then reload the
-`nginx` service (`service nginx reload`).
+    With older Zulip, you need to edit
+    `/etc/nginx/sites-available/zulip-enterprise` to comment out the
+    `nginx` configuration block for `/user_avatars` and the `include
+    /etc/nginx/zulip-include/uploads.route` line and then reload the
+    `nginx` service (`service nginx reload`).
 
 1. Finally, restart the Zulip server so that your settings changes
    take effect

docs/subsystems/full-text-search.md

@@ -77,7 +77,7 @@ messages until the migration finishes.
 
 Once the migrations are complete, restart Zulip:
 
-    su zulip -c /home/zulip/deployments/current/scripts/restart-server
+    su zulip -c '/home/zulip/deployments/current/scripts/restart-server'
 
 Now, you can use full-text search across all languages.
 
@@ -100,7 +100,7 @@ Then, set `USING_PGROONGA = False` in `/etc/zulip/settings.py`:
 
 And, restart Zulip:
 
-    su zulip -c /home/zulip/deployments/current/scripts/restart-server
+    su zulip -c '/home/zulip/deployments/current/scripts/restart-server'
 
 Now, full-text search feature based on PGroonga is disabled.  If you'd
 like, you can also remove the `pgroonga = enabled` line in

manage.py

@@ -1,16 +1,20 @@
 #!/usr/bin/env python3
+from __future__ import (print_function)
 import os
 import sys
 import types
+if sys.version_info <= (3, 0):
+    print("Error: Zulip is a Python 3 project, and cannot be run with Python 2.")
+    print("Use e.g. `/path/to/manage.py` not `python /path/to/manage.py`.")
+    sys.exit(1)
 
 BASE_DIR = os.path.dirname(os.path.abspath(__file__))
 sys.path.append(BASE_DIR)
 import scripts.lib.setup_path_on_import
+from scripts.lib.zulip_tools import script_should_not_be_root
 
 if __name__ == "__main__":
-    if 'posix' in os.name and os.geteuid() == 0:
-        print("manage.py should not be run as root.  Use `su zulip` to drop root.")
-        sys.exit(1)
+    script_should_not_be_root()
     if (os.access('/etc/zulip/zulip.conf', os.R_OK) and not
             os.access('/etc/zulip/zulip-secrets.conf', os.R_OK)):
         # The best way to detect running manage.py as another user in

puppet/zulip/files/nginx/zulip-include-frontend/app

@@ -60,14 +60,28 @@ location / {
     uwsgi_pass django;
 }
 
+# Certain Django routes not under /api are shared between mobile and
+# web and thus need API headers added.  We don't collapse this with the
+# above block for /events, because regular expressions take priority over
+# paths in nginx's order-of-operations, and we don't want to override the
+# tornado stuff.
+location ~ ^/(user_uploads|avatar|thumbnail)/ {
+    add_header Access-Control-Allow-Origin *;
+    add_header Access-Control-Allow-Headers Authorization;
+    add_header Access-Control-Allow-Methods 'GET, POST, DELETE, PUT, PATCH, HEAD';
+
+    include uwsgi_params;
+    uwsgi_pass django;
+}
+
+# Send all API routes not covered above to Django via uWSGI
 location /api/ {
     add_header Access-Control-Allow-Origin *;
     add_header Access-Control-Allow-Headers Authorization;
     add_header Access-Control-Allow-Methods 'GET, POST, DELETE, PUT, PATCH, HEAD';
 
     include uwsgi_params;
-    uwsgi_pass  django;
-
+    uwsgi_pass django;
 }
 
 include /etc/nginx/zulip-include/app.d/*.conf;

scripts/lib/clean_emoji_cache.py

@@ -35,7 +35,8 @@ def get_caches_in_use(threshold_days):
             # This happens for a deployment directory extracted from a
             # tarball, which just has a copy of the emoji data, not a symlink.
             continue
-        caches_in_use.add(os.readlink(emoji_link_path))
+        # The actual cache path doesn't include the /emoji
+        caches_in_use.add(os.path.dirname(os.readlink(emoji_link_path)))
     return caches_in_use
 
 def main(args: argparse.Namespace) -> None:

scripts/lib/clean_node_cache.py

@@ -45,7 +45,8 @@ def get_caches_in_use(threshold_days):
             # If 'package.json' file doesn't exist then no node_modules
             # cache is associated with this setup.
             continue
-        caches_in_use.add(os.readlink(node_modules_link_path))
+        # The actual cache path doesn't include the /node_modules
+        caches_in_use.add(os.path.dirname(os.readlink(node_modules_link_path)))
 
     return caches_in_use
 

scripts/lib/install

@@ -56,6 +56,7 @@ PUPPET_CLASSES="${PUPPET_CLASSES:-zulip::voyager}"
 VIRTUALENV_NEEDED="${VIRTUALENV_NEEDED:-yes}"
 
 if [ -n "$SELF_SIGNED_CERT" ] && [ -n "$USE_CERTBOT" ]; then
+    set +x
     echo "error: --self-signed-cert and --certbot are incompatible" >&2
     echo >&2
     usage
@@ -80,14 +81,16 @@ export LANGUAGE="en_US.UTF-8"
 
 # Check for a supported OS release.
 apt-get install -y lsb-release sudo
-os_release="$(lsb_release -sc)"
-case "$os_release" in
+os_info="$(lsb_release --short --id --release --codename)"
+{ read -r os_id; read -r os_release; read -r os_codename; } <<< "$os_info"
+
+case "$os_codename" in
     trusty|xenial|stretch|bionic) ;;
     *)
         set +x
         cat <<EOF
 
-Unsupported OS release: $os_release
+Unsupported OS release: $os_codename
 
 Zulip in production is supported only on:
  - Debian 9 "stretch"
@@ -101,12 +104,33 @@ EOF
         exit 1
 esac
 
+if [ "$os_id" = Ubuntu ] && ! apt-cache policy |
+           grep -q "^     release v=$os_release,o=Ubuntu,a=$os_codename,n=$os_codename,l=Ubuntu,c=universe"; then
+    set +x
+    cat <<'EOF'
+
+You must enable the Ubuntu Universe repository before installing
+Zulip.  You can do this with:
+
+    sudo add-apt-repository universe
+    sudo apt update
+
+For more information, see:
+  https://zulip.readthedocs.io/en/latest/production/requirements.html
+EOF
+    exit 1
+fi
+
 # Check for at least ~1.9GB of RAM before starting installation;
 # otherwise users will find out about insufficient RAM via weird
 # errors like a segfault running `pip install`.
 mem_kb=$(head -n1 /proc/meminfo | awk '{print $2}')
 if [ "$mem_kb" -lt 1900000 ]; then
-    echo "Insufficient RAM.  Zulip requires at least 2GB of RAM."
+    set +x
+    echo -e '\033[0;31m' >&2
+    echo "Insufficient RAM.  Zulip requires at least 2GB of RAM." >&2
+    echo >&2
+    echo -e '\033[0m' >&2
     exit 1
 fi
 
@@ -141,15 +165,22 @@ EOF
 fi
 
 apt-get -y dist-upgrade "${APT_OPTIONS[@]}"
-apt-get install -y \
+if ! apt-get install -y \
     puppet git curl wget \
     python python3 python-six python3-six crudini \
-    "${ADDITIONAL_PACKAGES[@]}"
+    "${ADDITIONAL_PACKAGES[@]}"; then
+    set +x
+    echo -e '\033[0;31m' >&2
+    echo "Installing packages failed; is network working and (on Ubuntu) the universe repository enabled?" >&2
+    echo >&2
+    echo -e '\033[0m' >&2
+    exit 1
+fi
 
 if [ -n "$USE_CERTBOT" ]; then
     "$ZULIP_PATH"/scripts/setup/setup-certbot \
         --no-zulip-conf --method=standalone \
-        --hostname "$EXTERNAL_HOST" --email "$ZULIP_ADMINISTRATOR"
+        "$EXTERNAL_HOST" --email "$ZULIP_ADMINISTRATOR"
 elif [ -n "$SELF_SIGNED_CERT" ]; then
     "$ZULIP_PATH"/scripts/setup/generate-self-signed-cert \
         --exists-ok "${EXTERNAL_HOST:-$(hostname)}"
@@ -303,7 +334,7 @@ if [ "$has_appserver" = 0 ]; then
         # If we're installing from a git checkout, we need to run
         # `tools/update-prod-static` in order to build the static
         # assets.
-        su zulip -c "/home/zulip/deployments/current/tools/update-prod-static --authors-not-required"
+        su zulip -c '/home/zulip/deployments/current/tools/update-prod-static --authors-not-required'
     fi
 fi
 
@@ -319,7 +350,7 @@ if [ -n "$NO_INIT_DB" ]; then
 
  Stopping because --no-init-db was passed.  To complete the installation, run:
 
-   su zulip -c /home/zulip/deployments/current/scripts/setup/initialize-database
+   su zulip -c '/home/zulip/deployments/current/scripts/setup/initialize-database'
 EOF
     exit 0
 fi

scripts/lib/setup-apt-repo

@@ -5,7 +5,7 @@ SOURCES_FILE=/etc/apt/sources.list.d/zulip.list
 STAMP_FILE=/etc/apt/sources.list.d/zulip.list.apt-update-in-progress
 zulip_source_hash=$(sha1sum "$SOURCES_FILE")
 
-apt-get install -y lsb-release apt-transport-https
+apt-get install -y lsb-release apt-transport-https gnupg
 
 SCRIPTS_PATH="$(dirname "$(dirname "$0")")"
 

scripts/lib/upgrade-zulip

@@ -11,16 +11,14 @@ os.environ["PYTHONUNBUFFERED"] = "y"
 
 sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..'))
 from scripts.lib.zulip_tools import DEPLOYMENTS_DIR, FAIL, WARNING, ENDC, \
-    su_to_zulip, get_deployment_lock, release_deployment_lock
+    su_to_zulip, get_deployment_lock, release_deployment_lock, script_should_be_root
+
+script_should_be_root(strip_lib_from_paths=True)
 
 logging.Formatter.converter = time.gmtime
 logging.basicConfig(format="%(asctime)s upgrade-zulip: %(message)s",
                     level=logging.INFO)
 
-if os.getuid() != 0:
-    logging.error("Must be run as root.")
-    sys.exit(1)
-
 if len(sys.argv) != 2:
     print(FAIL + "Usage: %s <tarball>" % (sys.argv[0],) + ENDC)
     sys.exit(1)

scripts/lib/upgrade-zulip-from-git

@@ -24,16 +24,14 @@ os.environ["PYTHONUNBUFFERED"] = "y"
 
 sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..'))
 from scripts.lib.zulip_tools import DEPLOYMENTS_DIR, FAIL, WARNING, ENDC, make_deploy_path, \
-    get_deployment_lock, release_deployment_lock, su_to_zulip
+    get_deployment_lock, release_deployment_lock, su_to_zulip, script_should_be_root
+
+script_should_be_root(strip_lib_from_paths=True)
 
 logging.Formatter.converter = time.gmtime
 logging.basicConfig(format="%(asctime)s upgrade-zulip-from-git: %(message)s",
                     level=logging.INFO)
 
-if os.getuid() != 0:
-    logging.error("Must be run as root.")
-    sys.exit(1)
-
 parser = argparse.ArgumentParser()
 parser.add_argument("refname", help="Git reference, e.g. a branch, tag, or commit ID.")
 parser.add_argument("--remote-url", dest="remote_url",

scripts/lib/upgrade-zulip-stage-2

@@ -20,16 +20,14 @@ os.environ["LANG"] = "en_US.UTF-8"
 os.environ["LANGUAGE"] = "en_US.UTF-8"
 
 sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..'))
-from scripts.lib.zulip_tools import DEPLOYMENTS_DIR, FAIL, WARNING, ENDC, su_to_zulip
+from scripts.lib.zulip_tools import DEPLOYMENTS_DIR, FAIL, WARNING, ENDC, su_to_zulip, script_should_be_root
+
+script_should_be_root()
 
 logging.Formatter.converter = time.gmtime
 logging.basicConfig(format="%(asctime)s upgrade-zulip-stage-2: %(message)s",
                     level=logging.INFO)
 
-if os.getuid() != 0:
-    logging.error("Must be run as root.")
-    sys.exit(1)
-
 # make sure we have appropriate file permissions
 os.umask(0o22)
 
@@ -42,6 +40,8 @@ parser.add_argument("--skip-migrations", dest="skip_migrations", action='store_t
                     help="Skip doing migrations.")
 parser.add_argument("--from-git", dest="from_git", action='store_true',
                     help="Upgrading from git, so run update-prod-static.")
+parser.add_argument("--skip-purge-old-deployments", dest="skip_purge_old_deployments",
+                    action="store_true", help="Skip purging old deployments.")
 args = parser.parse_args()
 
 deploy_path = args.deploy_path
@@ -170,4 +170,8 @@ logging.info("Restarting Zulip...")
 subprocess.check_output(["./scripts/restart-server"], preexec_fn=su_to_zulip)
 logging.info("Upgrade complete!")
 
-subprocess.check_call(["./scripts/purge-old-deployments"])
+if not args.skip_purge_old_deployments:
+    logging.info("Purging old deployments...")
+    subprocess.check_call(["./scripts/purge-old-deployments"])
+else:
+    logging.info("Skipping purging old deployments.")

scripts/lib/zulip_tools.py

@@ -359,3 +359,30 @@ def file_or_package_hash_updated(paths, hash_name, is_force, package_versions=[]
             hash_file.write(new_hash)
             return True
     return False
+
+def is_root() -> bool:
+    if 'posix' in os.name and os.geteuid() == 0:
+        return True
+    return False
+
+def script_should_not_be_root() -> None:
+    script_name = os.path.abspath(sys.argv[0])
+    if is_root():
+        msg = ("{shortname} should not be run as root. Use `su zulip` to switch to the 'zulip'\n"
+               "user before rerunning this, or use \n  su zulip -c '{name} ...'\n"
+               "to switch users and run this as a single command.").format(
+            name=script_name,
+            shortname=os.path.basename(script_name))
+        print(msg)
+        sys.exit(1)
+
+def script_should_be_root(strip_lib_from_paths: bool=False) -> None:
+    script_name = os.path.abspath(sys.argv[0])
+    # Since these Python scripts are run inside a thin shell wrapper,
+    # we need to replace the paths in order to ensure we instruct
+    # users to (re)run the right command.
+    if strip_lib_from_paths:
+        script_name = script_name.replace("scripts/lib/upgrade", "scripts/upgrade")
+    if not is_root():
+        print("{} must be run as root.".format(script_name))
+        sys.exit(1)

scripts/setup/setup-certbot

@@ -4,7 +4,8 @@ set -e
 
 usage() {
     cat <<EOF >&2
-Usage: $0 --hostname=zulip.example.com --email=admin@example.com [--method={webroot|standalone}] [--no-zulip-conf]
+Usage: $0 --email=admin@example.com [--method={webroot|standalone}] \
+[--no-zulip-conf] hostname.example.com [another.example.com]
 EOF
     exit 1
 }
@@ -15,15 +16,10 @@ if [ "$EUID" -ne 0 ]; then
 fi
 
 method=webroot
-args="$(getopt -o '' --long help,hostname:,email:,method:,deploy-hook:,no-zulip-conf,agree-tos -n "$0" -- "$@")"
+args="$(getopt -o '' --long help,email:,method:,deploy-hook:,no-zulip-conf,agree-tos -n "$0" -- "$@")"
 eval "set -- $args"
 while true; do
     case "$1" in
-        --hostname)
-            DOMAIN="$2"
-            shift
-            shift
-            ;;
         --email)
             EMAIL="$2"
             shift
@@ -52,11 +48,19 @@ while true; do
             shift
             ;;
         --)
+            shift
             break
             ;;
     esac
 done
 
+# Parse the remaining arguments as Subject Alternative Names to pass to certbot
+HOSTNAMES=()
+for arg; do
+    HOSTNAMES+=(-d "$arg")
+done
+DOMAIN=$1
+
 if [ -n "$show_help" ]; then
     usage
 fi
@@ -94,7 +98,7 @@ chmod a+x "$CERTBOT_PATH"
 # Passing --force-interactive suppresses a warning, but also brings up
 # an annoying prompt we stifle with --no-eff-email.
 "$CERTBOT_PATH" certonly "${method_args[@]}" \
-                -d "$DOMAIN" -m "$EMAIL" \
+                "${HOSTNAMES[@]}" -m "$EMAIL" \
                 $agree_tos --force-renewal \
                 "${deploy_hook[@]}" \
                 --force-interactive --no-eff-email

scripts/zulip-puppet-apply

@@ -5,7 +5,9 @@ import sys
 import subprocess
 import configparser
 import re
-from lib.zulip_tools import parse_lsb_release
+from lib.zulip_tools import parse_lsb_release, script_should_be_root
+
+script_should_be_root()
 
 force = False
 extra_args = sys.argv[1:]

static/locale/gl/LC_MESSAGES/django.po

@@ -8,7 +8,7 @@ msgid ""
 msgstr ""
 "Project-Id-Version: PACKAGE VERSION\n"
 "Report-Msgid-Bugs-To: \n"
-"POT-Creation-Date: 2018-11-07 15:19+0000\n"
+"POT-Creation-Date: 2018-11-28 20:32+0000\n"
 "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
 "Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
 "Language-Team: LANGUAGE <LL@li.org>\n"
@@ -1393,9 +1393,10 @@ msgstr ""
 
 #: templates/zerver/emails/compiled/confirm_new_email.html:29
 #: templates/zerver/emails/compiled/confirm_registration.html:27
-#: templates/zerver/emails/compiled/followup_day1.html:56
+#: templates/zerver/emails/compiled/followup_day1.html:55
 #: templates/zerver/emails/compiled/invitation.html:26
 #: templates/zerver/emails/compiled/invitation_reminder.html:22
+#: templates/zerver/emails/compiled/realm_reactivation.html:36
 #: templates/zerver/emails/confirm_new_email.source.html:28
 #: templates/zerver/emails/confirm_new_email.txt:15
 #: templates/zerver/emails/confirm_registration.source.html:26
@@ -1411,8 +1412,9 @@ msgstr ""
 
 #: templates/zerver/emails/compiled/confirm_new_email.html:30
 #: templates/zerver/emails/compiled/confirm_registration.html:28
-#: templates/zerver/emails/compiled/followup_day1.html:57
+#: templates/zerver/emails/compiled/followup_day1.html:56
 #: templates/zerver/emails/compiled/notify_change_in_email.html:15
+#: templates/zerver/emails/compiled/realm_reactivation.html:37
 #: templates/zerver/emails/confirm_new_email.source.html:29
 #: templates/zerver/emails/confirm_new_email.txt:16
 #: templates/zerver/emails/confirm_registration.source.html:27
@@ -1452,6 +1454,7 @@ msgid "Complete registration"
 msgstr ""
 
 #: templates/zerver/emails/compiled/confirm_registration.html:20
+#: templates/zerver/emails/compiled/realm_reactivation.html:29
 #, python-format
 msgid ""
 "\n"
@@ -1500,88 +1503,80 @@ msgid "Thanks for using Zulip!"
 msgstr ""
 
 #: templates/zerver/emails/compiled/followup_day1.html:9
-#: templates/zerver/emails/compiled/invitation.html:9
-#: templates/zerver/emails/followup_day1.source.html:8
-#: templates/zerver/emails/followup_day1.txt:1
-#: templates/zerver/emails/invitation.source.html:8
-#: templates/zerver/emails/invitation.txt:1
-msgid "Hi there,"
+msgid "Welcome to Zulip!"
 msgstr ""
 
-#: templates/zerver/emails/compiled/followup_day1.html:11
-#: templates/zerver/emails/followup_day1.source.html:10
-#: templates/zerver/emails/followup_day1.txt:3
-msgid "Welcome to Zulip! A few tips to get you started:"
-msgstr ""
-
-#: templates/zerver/emails/compiled/followup_day1.html:14
+#: templates/zerver/emails/compiled/followup_day1.html:13
 #, python-format
 msgid ""
 "\n"
-"    Zulip works best when it's always open, so we suggest downloading\n"
-"    our <a href=\"https://zulipchat.com/apps\" style=\"color:hsl(164, 42%%, "
-"47%%); text-decoration:underline\">desktop and mobile apps</a>.\n"
-"    "
+"        You've created the new Zulip organization <b>%(realm_name)s</b>.\n"
+"        "
 msgstr ""
 
-#: templates/zerver/emails/compiled/followup_day1.html:19
-#: templates/zerver/emails/followup_day1.source.html:18
+#: templates/zerver/emails/compiled/followup_day1.html:17
+#, python-format
 msgid ""
 "\n"
-"    To access your account from the apps, enter this Organization URL:\n"
-"    "
+"        You've joined the Zulip organization <b>%(realm_name)s</b>.\n"
+"        "
+msgstr ""
+
+#: templates/zerver/emails/compiled/followup_day1.html:24
+msgid "Your account details:"
+msgstr ""
+
+#: templates/zerver/emails/compiled/followup_day1.html:26
+msgid "Organization URL:"
 msgstr ""
 
 #: templates/zerver/emails/compiled/followup_day1.html:29
-#, python-format
-msgid ""
-"\n"
-"    Become a Zulip pro with a few\n"
-"    <a href=\"%(keyboard_shortcuts_link)s\" style=\"color:hsl(164, 42%%, "
-"47%%); text-decoration:underline\">keyboard shortcuts</a>:\n"
-"    "
+msgid "Use your LDAP account to login"
 msgstr ""
 
-#: templates/zerver/emails/compiled/followup_day1.html:34
-#: templates/zerver/emails/followup_day1.source.html:33
-#: templates/zerver/emails/followup_day1.txt:17
-msgid "Next unread thread"
+#: templates/zerver/emails/compiled/followup_day1.html:32
+msgid "Email:"
 msgstr ""
 
 #: templates/zerver/emails/compiled/followup_day1.html:35
-#: templates/zerver/emails/followup_day1.source.html:34
-#: templates/zerver/emails/followup_day1.txt:18
-msgid "Reply to message under the blue box"
-msgstr ""
-
-#: templates/zerver/emails/compiled/followup_day1.html:36
-#: templates/zerver/emails/followup_day1.source.html:35
-#: templates/zerver/emails/followup_day1.txt:19
-msgid "Start a new topic"
-msgstr ""
-
-#: templates/zerver/emails/compiled/followup_day1.html:40
 #, python-format
 msgid ""
 "\n"
-"    Give our <a href=\"%(getting_started_link)s\" style=\"color:hsl(164, "
-"42%%, 47%%); text-decoration:underline\">guide for new\n"
-"    %(user_role_group)s</a> a spin.\n"
+"    (you'll need these to sign in to the <a href=\"https://zulipchat.com/apps"
+"\" style=\"color:hsl(164, 42%%, 47%%); text-decoration:underline\">mobile "
+"and desktop</a> apps)\n"
 "    "
 msgstr ""
 
+#: templates/zerver/emails/compiled/followup_day1.html:42
+#, python-format
+msgid ""
+"\n"
+"        Check out our <a href=\"%(getting_started_link)s\" style=\"color:"
+"hsl(164, 42%%, 47%%); text-decoration:underline\">guide for admins</a>, "
+"become a Zulip pro with a\n"
+"        few <a href=\"%(keyboard_shortcuts_link)s\" style=\"color:hsl(164, "
+"42%%, 47%%); text-decoration:underline\">keyboard shortcuts</a>, or <a href="
+"\"%(realm_uri)s\" style=\"color:hsl(164, 42%%, 47%%); text-decoration:"
+"underline\">dive right in</a>!\n"
+"        "
+msgstr ""
+
 #: templates/zerver/emails/compiled/followup_day1.html:47
+#, python-format
 msgid ""
 "\n"
-"    Zulip combines the real-time ease of chat with the threaded "
-"organization\n"
-"    of email. Zulip is about productivity—making communication fun and\n"
-"    easy, while avoiding the distracting and disorganized conversations of\n"
-"    chatrooms. We hope you love using Zulip as much as we do.\n"
-"    "
+"        <a href=\"%(getting_started_link)s\" style=\"color:hsl(164, 42%%, "
+"47%%); text-decoration:underline\">Learn more</a> about Zulip, become a pro "
+"with a few\n"
+"        <a href=\"%(keyboard_shortcuts_link)s\" style=\"color:hsl(164, 42%%, "
+"47%%); text-decoration:underline\">keyboard shortcuts</a>, or <a href="
+"\"%(realm_uri)s\" style=\"color:hsl(164, 42%%, 47%%); text-decoration:"
+"underline\">dive right in</a>!\n"
+"        "
 msgstr ""
 
-#: templates/zerver/emails/compiled/followup_day1.html:61
+#: templates/zerver/emails/compiled/followup_day1.html:60
 #, python-format
 msgid ""
 "\n"
@@ -1595,6 +1590,14 @@ msgid ""
 "    "
 msgstr ""
 
+#: templates/zerver/emails/compiled/invitation.html:9
+#: templates/zerver/emails/followup_day1.source.html:8
+#: templates/zerver/emails/followup_day1.txt:1
+#: templates/zerver/emails/invitation.source.html:8
+#: templates/zerver/emails/invitation.txt:1
+msgid "Hi there,"
+msgstr ""
+
 #: templates/zerver/emails/compiled/invitation.html:12
 #, python-format
 msgid ""
@@ -1675,6 +1678,38 @@ msgstr ""
 msgid "Best,"
 msgstr ""
 
+#: templates/zerver/emails/compiled/realm_reactivation.html:10
+#, python-format
+msgid ""
+"\n"
+"    Dear former administrators of %(realm_name)s,\n"
+"    "
+msgstr ""
+
+#: templates/zerver/emails/compiled/realm_reactivation.html:15
+#, python-format
+msgid ""
+"\n"
+"    One of your administrators requested reactivation of the\n"
+"    previously deactivated Zulip organization hosted at %(realm_uri)s.  If "
+"you'd\n"
+"    like to do confirm that request and reactivate the organization, please "
+"click here:\n"
+"    "
+msgstr ""
+
+#: templates/zerver/emails/compiled/realm_reactivation.html:21
+msgid "Reactivate organization"
+msgstr ""
+
+#: templates/zerver/emails/compiled/realm_reactivation.html:23
+msgid ""
+"\n"
+"    If the request was in error, you can take no action and this link\n"
+"    will expire in 24 hours.\n"
+"    "
+msgstr ""
+
 #: templates/zerver/emails/confirm_new_email.source.html:21
 #, python-format
 msgid ""
@@ -1719,6 +1754,11 @@ msgid ""
 "questions."
 msgstr ""
 
+#: templates/zerver/emails/followup_day1.source.html:10
+#: templates/zerver/emails/followup_day1.txt:3
+msgid "Welcome to Zulip! A few tips to get you started:"
+msgstr ""
+
 #: templates/zerver/emails/followup_day1.source.html:13
 msgid ""
 "\n"
@@ -1727,6 +1767,13 @@ msgid ""
 "    "
 msgstr ""
 
+#: templates/zerver/emails/followup_day1.source.html:18
+msgid ""
+"\n"
+"    To access your account from the apps, enter this Organization URL:\n"
+"    "
+msgstr ""
+
 #: templates/zerver/emails/followup_day1.source.html:28
 #, python-format
 msgid ""
@@ -1736,6 +1783,21 @@ msgid ""
 "    "
 msgstr ""
 
+#: templates/zerver/emails/followup_day1.source.html:33
+#: templates/zerver/emails/followup_day1.txt:17
+msgid "Next unread thread"
+msgstr ""
+
+#: templates/zerver/emails/followup_day1.source.html:34
+#: templates/zerver/emails/followup_day1.txt:18
+msgid "Reply to message under the blue box"
+msgstr ""
+
+#: templates/zerver/emails/followup_day1.source.html:35
+#: templates/zerver/emails/followup_day1.txt:19
+msgid "Start a new topic"
+msgstr ""
+
 #: templates/zerver/emails/followup_day1.source.html:39
 #, python-format
 msgid ""
@@ -3324,43 +3386,43 @@ msgstr ""
 msgid "Wrong subdomain"
 msgstr ""
 
-#: zerver/views/auth.py:716 zerver/views/auth.py:747
+#: zerver/views/auth.py:717 zerver/views/auth.py:748
 msgid "Dev environment not enabled."
 msgstr ""
 
-#: zerver/views/auth.py:732 zerver/views/auth.py:780
+#: zerver/views/auth.py:733 zerver/views/auth.py:781
 msgid "This organization has been deactivated."
 msgstr ""
 
-#: zerver/views/auth.py:735 zerver/views/auth.py:777
+#: zerver/views/auth.py:736 zerver/views/auth.py:778
 msgid "Your account has been disabled."
 msgstr ""
 
-#: zerver/views/auth.py:738
+#: zerver/views/auth.py:739
 msgid "This user is not registered."
 msgstr ""
 
-#: zerver/views/auth.py:783
+#: zerver/views/auth.py:784
 msgid "Password auth is disabled in your team."
 msgstr ""
 
-#: zerver/views/auth.py:789
+#: zerver/views/auth.py:790
 msgid "This user is not registered; do so from a browser."
 msgstr ""
 
-#: zerver/views/auth.py:791 zerver/views/auth.py:875
+#: zerver/views/auth.py:792 zerver/views/auth.py:876
 msgid "Your username or password is incorrect."
 msgstr ""
 
-#: zerver/views/auth.py:816
+#: zerver/views/auth.py:817
 msgid "Invalid subdomain"
 msgstr ""
 
-#: zerver/views/auth.py:822
+#: zerver/views/auth.py:823
 msgid "Subdomain required"
 msgstr ""
 
-#: zerver/views/auth.py:883
+#: zerver/views/auth.py:884
 msgid "GOOGLE_CLIENT_ID is not configured"
 msgstr ""
 

static/locale/ko/translations.json

@@ -2,11 +2,11 @@
   "\"__file_name__\" was too large; the maximum file size is 25MiB.": "\"__file_name__\" 이 너무 큽니다.; 최대 파일크기는 25MiB입니다.",
   "(This user has been deactivated)": "(이 사용자는 비활성화되었습니다.)",
   "(no topic)": "(주제없음)",
-  "1 day": "",
-  "1 hour": "",
-  "1 week": "",
-  "10 minutes": "",
-  "2 minutes": "",
+  "1 day": "1 일",
+  "1 hour": "1 시간",
+  "1 week": "1 주",
+  "10 minutes": "10 분",
+  "2 minutes": "2 분",
   "24-hour time (17:00 instead of 5:00 PM)": "24 시간 표시 (5:00 PM 대신 17:00)",
   "<b>Private, protected history:</b> must be invited by a member; new members can only see messages sent after they join; hidden from non-administrator users": "",
   "<b>Private, shared history:</b> must be invited by a member; new members can view complete message history; hidden from non-administrator users": "",
@@ -39,13 +39,13 @@
   "Add member...": "회원 추가 ...",
   "Add members of your organization to mentionable user groups.": "관심있는 사용자 그룹에 단체 구성원을 추가하십시오.",
   "Add new default stream": "새 기본 스트림 추가",
-  "Add option": "",
+  "Add option": "옵션 추가",
   "Add profile field": "",
-  "Add question": "",
+  "Add question": "질문 추가",
   "Add stream": "스트림 추가",
-  "Add task": "",
+  "Add task": "업무 추가",
   "Added successfully!": "성공적으로 추가됨!",
-  "Administrator": "",
+  "Administrator": "관리자",
   "Administrators can always delete any message.": "관리자는 언제든지 모든 메시지를 삭제할 수 있습니다.",
   "Admins only": "관리자 만",
   "Alert word": "경고문",
@@ -149,7 +149,7 @@
   "Default language": "기본 언어",
   "Default streams": "기본 스트림",
   "Default user settings": "",
-  "Delete": "",
+  "Delete": "삭제",
   "Delete alert word": "경고문 삭제",
   "Delete avatar": "아바타 삭제",
   "Delete bot": "봇 삭제",

static/locale/mobile_info.json

@@ -44,7 +44,7 @@
     "total": 140
   },
   "it": {
-    "not_translated": 129,
+    "not_translated": 0,
     "total": 140
   },
   "ja": {
@@ -52,7 +52,7 @@
     "total": 140
   },
   "ko": {
-    "not_translated": 10,
+    "not_translated": 9,
     "total": 140
   },
   "ml": {
@@ -92,7 +92,7 @@
     "total": 140
   },
   "zh_Hans": {
-    "not_translated": 8,
+    "not_translated": 0,
     "total": 140
   },
   "zh_Hant": {

static/locale/pt/LC_MESSAGES/django.po

@@ -13,7 +13,7 @@ msgstr ""
 "Project-Id-Version: Zulip\n"
 "Report-Msgid-Bugs-To: \n"
 "POT-Creation-Date: 2018-11-03 00:32+0000\n"
-"PO-Revision-Date: 2018-11-05 14:34+0000\n"
+"PO-Revision-Date: 2018-11-07 18:54+0000\n"
 "Last-Translator: André Lopes Pereira <andrelopespereira@gmail.com>\n"
 "Language-Team: Portuguese (http://www.transifex.com/zulip/zulip/language/pt/)\n"
 "MIME-Version: 1.0\n"
@@ -842,7 +842,7 @@ msgstr "Documentação detalhada de atalhos de teclado"
 #: templates/zerver/app/left_sidebar.html:5
 #: templates/zerver/app/left_sidebar.html:10
 msgid "All messages"
-msgstr "Todas mensagens"
+msgstr "Todas as mensagens"
 
 #: templates/zerver/app/left_sidebar.html:33
 msgid "Starred messages"

static/locale/pt/translations.json

@@ -276,7 +276,7 @@
   "Mark all messages in <b>__stream.name__</b> as read": "Marcar todas as mensagens em <b>__stream.name__</b> como lidas",
   "Mark all messages in <b>__topic_name__</b> as read": "Marcar todas as mensagens em <b>__topic_name__</b> como lidas",
   "Marketing team": "Time de marketing",
-  "Marking all messages as read\u2026": "Marcando todas as mensagens como lidas...",
+  "Marking all messages as read\u2026": "Marcando todas as mensagens como lidas\\\\u2026",
   "Member": "Membro",
   "Members and admins": "Membros e administradores",
   "Members and admins, but only admins can add generic bots": "Membros e administradores, mas apenas administradores podem adicionar bots genéricos",

static/locale/ru/LC_MESSAGES/django.po

@@ -11,7 +11,7 @@
 # Sergey Korablin <s.korablin@gmail.com>, 2018
 # Sergey Korablin <s.korablin@gmail.com>, 2018
 # Никита Радченко <aygolan@gmail.com>, 2016
-# Султонбек Ахмедов <cooltonbek@gmail.com>, 2018
+# Султонбек Ахмедов <davlaterra@ya.ru>, 2018
 msgid ""
 msgstr ""
 "Project-Id-Version: Zulip\n"

static/locale/ta/LC_MESSAGES/django.po

@@ -8,7 +8,7 @@ msgid ""
 msgstr ""
 "Project-Id-Version: Zulip\n"
 "Report-Msgid-Bugs-To: \n"
-"POT-Creation-Date: 2018-11-07 15:19+0000\n"
+"POT-Creation-Date: 2018-11-28 20:32+0000\n"
 "PO-Revision-Date: 2018-04-11 21:06+0000\n"
 "Last-Translator: Tim Abbott <tabbott@kandralabs.com>\n"
 "Language-Team: Tamil (http://www.transifex.com/zulip/zulip/language/ta/)\n"
@@ -1407,9 +1407,10 @@ msgstr ""
 
 #: templates/zerver/emails/compiled/confirm_new_email.html:29
 #: templates/zerver/emails/compiled/confirm_registration.html:27
-#: templates/zerver/emails/compiled/followup_day1.html:56
+#: templates/zerver/emails/compiled/followup_day1.html:55
 #: templates/zerver/emails/compiled/invitation.html:26
 #: templates/zerver/emails/compiled/invitation_reminder.html:22
+#: templates/zerver/emails/compiled/realm_reactivation.html:36
 #: templates/zerver/emails/confirm_new_email.source.html:28
 #: templates/zerver/emails/confirm_new_email.txt:15
 #: templates/zerver/emails/confirm_registration.source.html:26
@@ -1425,8 +1426,9 @@ msgstr ""
 
 #: templates/zerver/emails/compiled/confirm_new_email.html:30
 #: templates/zerver/emails/compiled/confirm_registration.html:28
-#: templates/zerver/emails/compiled/followup_day1.html:57
+#: templates/zerver/emails/compiled/followup_day1.html:56
 #: templates/zerver/emails/compiled/notify_change_in_email.html:15
+#: templates/zerver/emails/compiled/realm_reactivation.html:37
 #: templates/zerver/emails/confirm_new_email.source.html:29
 #: templates/zerver/emails/confirm_new_email.txt:16
 #: templates/zerver/emails/confirm_registration.source.html:27
@@ -1468,6 +1470,7 @@ msgid "Complete registration"
 msgstr ""
 
 #: templates/zerver/emails/compiled/confirm_registration.html:20
+#: templates/zerver/emails/compiled/realm_reactivation.html:29
 #, python-format
 msgid ""
 "\n"
@@ -1516,88 +1519,82 @@ msgid "Thanks for using Zulip!"
 msgstr ""
 
 #: templates/zerver/emails/compiled/followup_day1.html:9
-#: templates/zerver/emails/compiled/invitation.html:9
-#: templates/zerver/emails/followup_day1.source.html:8
-#: templates/zerver/emails/followup_day1.txt:1
-#: templates/zerver/emails/invitation.source.html:8
-#: templates/zerver/emails/invitation.txt:1
-msgid "Hi there,"
+msgid "Welcome to Zulip!"
 msgstr ""
 
-#: templates/zerver/emails/compiled/followup_day1.html:11
-#: templates/zerver/emails/followup_day1.source.html:10
-#: templates/zerver/emails/followup_day1.txt:3
-msgid "Welcome to Zulip! A few tips to get you started:"
-msgstr ""
-
-#: templates/zerver/emails/compiled/followup_day1.html:14
+#: templates/zerver/emails/compiled/followup_day1.html:13
 #, python-format
 msgid ""
 "\n"
-"    Zulip works best when it's always open, so we suggest downloading\n"
-"    our <a href=\"https://zulipchat.com/apps\" style=\"color:hsl(164, 42%%, "
-"47%%); text-decoration:underline\">desktop and mobile apps</a>.\n"
-"    "
+"        You've created the new Zulip organization <b>%(realm_name)s</b>.\n"
+"        "
 msgstr ""
 
-#: templates/zerver/emails/compiled/followup_day1.html:19
-#: templates/zerver/emails/followup_day1.source.html:18
+#: templates/zerver/emails/compiled/followup_day1.html:17
+#, python-format
 msgid ""
 "\n"
-"    To access your account from the apps, enter this Organization URL:\n"
-"    "
+"        You've joined the Zulip organization <b>%(realm_name)s</b>.\n"
+"        "
+msgstr ""
+
+#: templates/zerver/emails/compiled/followup_day1.html:24
+msgid "Your account details:"
+msgstr ""
+
+#: templates/zerver/emails/compiled/followup_day1.html:26
+msgid "Organization URL:"
 msgstr ""
 
 #: templates/zerver/emails/compiled/followup_day1.html:29
-#, python-format
-msgid ""
-"\n"
-"    Become a Zulip pro with a few\n"
-"    <a href=\"%(keyboard_shortcuts_link)s\" style=\"color:hsl(164, 42%%, "
-"47%%); text-decoration:underline\">keyboard shortcuts</a>:\n"
-"    "
+msgid "Use your LDAP account to login"
 msgstr ""
 
-#: templates/zerver/emails/compiled/followup_day1.html:34
-#: templates/zerver/emails/followup_day1.source.html:33
-#: templates/zerver/emails/followup_day1.txt:17
-msgid "Next unread thread"
-msgstr ""
+#: templates/zerver/emails/compiled/followup_day1.html:32
+#, fuzzy
+#| msgid "Email"
+msgid "Email:"
+msgstr "மின்னஞ்சல்"
 
 #: templates/zerver/emails/compiled/followup_day1.html:35
-#: templates/zerver/emails/followup_day1.source.html:34
-#: templates/zerver/emails/followup_day1.txt:18
-msgid "Reply to message under the blue box"
-msgstr ""
-
-#: templates/zerver/emails/compiled/followup_day1.html:36
-#: templates/zerver/emails/followup_day1.source.html:35
-#: templates/zerver/emails/followup_day1.txt:19
-msgid "Start a new topic"
-msgstr ""
-
-#: templates/zerver/emails/compiled/followup_day1.html:40
 #, python-format
 msgid ""
 "\n"
-"    Give our <a href=\"%(getting_started_link)s\" style=\"color:hsl(164, "
-"42%%, 47%%); text-decoration:underline\">guide for new\n"
-"    %(user_role_group)s</a> a spin.\n"
+"    (you'll need these to sign in to the <a href=\"https://zulipchat.com/apps"
+"\" style=\"color:hsl(164, 42%%, 47%%); text-decoration:underline\">mobile "
+"and desktop</a> apps)\n"
 "    "
 msgstr ""
 
+#: templates/zerver/emails/compiled/followup_day1.html:42
+#, python-format
+msgid ""
+"\n"
+"        Check out our <a href=\"%(getting_started_link)s\" style=\"color:"
+"hsl(164, 42%%, 47%%); text-decoration:underline\">guide for admins</a>, "
+"become a Zulip pro with a\n"
+"        few <a href=\"%(keyboard_shortcuts_link)s\" style=\"color:hsl(164, "
+"42%%, 47%%); text-decoration:underline\">keyboard shortcuts</a>, or <a href="
+"\"%(realm_uri)s\" style=\"color:hsl(164, 42%%, 47%%); text-decoration:"
+"underline\">dive right in</a>!\n"
+"        "
+msgstr ""
+
 #: templates/zerver/emails/compiled/followup_day1.html:47
+#, python-format
 msgid ""
 "\n"
-"    Zulip combines the real-time ease of chat with the threaded "
-"organization\n"
-"    of email. Zulip is about productivity—making communication fun and\n"
-"    easy, while avoiding the distracting and disorganized conversations of\n"
-"    chatrooms. We hope you love using Zulip as much as we do.\n"
-"    "
+"        <a href=\"%(getting_started_link)s\" style=\"color:hsl(164, 42%%, "
+"47%%); text-decoration:underline\">Learn more</a> about Zulip, become a pro "
+"with a few\n"
+"        <a href=\"%(keyboard_shortcuts_link)s\" style=\"color:hsl(164, 42%%, "
+"47%%); text-decoration:underline\">keyboard shortcuts</a>, or <a href="
+"\"%(realm_uri)s\" style=\"color:hsl(164, 42%%, 47%%); text-decoration:"
+"underline\">dive right in</a>!\n"
+"        "
 msgstr ""
 
-#: templates/zerver/emails/compiled/followup_day1.html:61
+#: templates/zerver/emails/compiled/followup_day1.html:60
 #, python-format
 msgid ""
 "\n"
@@ -1611,6 +1608,14 @@ msgid ""
 "    "
 msgstr ""
 
+#: templates/zerver/emails/compiled/invitation.html:9
+#: templates/zerver/emails/followup_day1.source.html:8
+#: templates/zerver/emails/followup_day1.txt:1
+#: templates/zerver/emails/invitation.source.html:8
+#: templates/zerver/emails/invitation.txt:1
+msgid "Hi there,"
+msgstr ""
+
 #: templates/zerver/emails/compiled/invitation.html:12
 #, python-format
 msgid ""
@@ -1691,6 +1696,38 @@ msgstr ""
 msgid "Best,"
 msgstr ""
 
+#: templates/zerver/emails/compiled/realm_reactivation.html:10
+#, python-format
+msgid ""
+"\n"
+"    Dear former administrators of %(realm_name)s,\n"
+"    "
+msgstr ""
+
+#: templates/zerver/emails/compiled/realm_reactivation.html:15
+#, python-format
+msgid ""
+"\n"
+"    One of your administrators requested reactivation of the\n"
+"    previously deactivated Zulip organization hosted at %(realm_uri)s.  If "
+"you'd\n"
+"    like to do confirm that request and reactivate the organization, please "
+"click here:\n"
+"    "
+msgstr ""
+
+#: templates/zerver/emails/compiled/realm_reactivation.html:21
+msgid "Reactivate organization"
+msgstr ""
+
+#: templates/zerver/emails/compiled/realm_reactivation.html:23
+msgid ""
+"\n"
+"    If the request was in error, you can take no action and this link\n"
+"    will expire in 24 hours.\n"
+"    "
+msgstr ""
+
 #: templates/zerver/emails/confirm_new_email.source.html:21
 #, python-format
 msgid ""
@@ -1735,6 +1772,11 @@ msgid ""
 "questions."
 msgstr ""
 
+#: templates/zerver/emails/followup_day1.source.html:10
+#: templates/zerver/emails/followup_day1.txt:3
+msgid "Welcome to Zulip! A few tips to get you started:"
+msgstr ""
+
 #: templates/zerver/emails/followup_day1.source.html:13
 msgid ""
 "\n"
@@ -1743,6 +1785,13 @@ msgid ""
 "    "
 msgstr ""
 
+#: templates/zerver/emails/followup_day1.source.html:18
+msgid ""
+"\n"
+"    To access your account from the apps, enter this Organization URL:\n"
+"    "
+msgstr ""
+
 #: templates/zerver/emails/followup_day1.source.html:28
 #, python-format
 msgid ""
@@ -1752,6 +1801,21 @@ msgid ""
 "    "
 msgstr ""
 
+#: templates/zerver/emails/followup_day1.source.html:33
+#: templates/zerver/emails/followup_day1.txt:17
+msgid "Next unread thread"
+msgstr ""
+
+#: templates/zerver/emails/followup_day1.source.html:34
+#: templates/zerver/emails/followup_day1.txt:18
+msgid "Reply to message under the blue box"
+msgstr ""
+
+#: templates/zerver/emails/followup_day1.source.html:35
+#: templates/zerver/emails/followup_day1.txt:19
+msgid "Start a new topic"
+msgstr ""
+
 #: templates/zerver/emails/followup_day1.source.html:39
 #, python-format
 msgid ""
@@ -3342,43 +3406,43 @@ msgstr ""
 msgid "Wrong subdomain"
 msgstr ""
 
-#: zerver/views/auth.py:716 zerver/views/auth.py:747
+#: zerver/views/auth.py:717 zerver/views/auth.py:748
 msgid "Dev environment not enabled."
 msgstr ""
 
-#: zerver/views/auth.py:732 zerver/views/auth.py:780
+#: zerver/views/auth.py:733 zerver/views/auth.py:781
 msgid "This organization has been deactivated."
 msgstr ""
 
-#: zerver/views/auth.py:735 zerver/views/auth.py:777
+#: zerver/views/auth.py:736 zerver/views/auth.py:778
 msgid "Your account has been disabled."
 msgstr ""
 
-#: zerver/views/auth.py:738
+#: zerver/views/auth.py:739
 msgid "This user is not registered."
 msgstr ""
 
-#: zerver/views/auth.py:783
+#: zerver/views/auth.py:784
 msgid "Password auth is disabled in your team."
 msgstr ""
 
-#: zerver/views/auth.py:789
+#: zerver/views/auth.py:790
 msgid "This user is not registered; do so from a browser."
 msgstr ""
 
-#: zerver/views/auth.py:791 zerver/views/auth.py:875
+#: zerver/views/auth.py:792 zerver/views/auth.py:876
 msgid "Your username or password is incorrect."
 msgstr ""
 
-#: zerver/views/auth.py:816
+#: zerver/views/auth.py:817
 msgid "Invalid subdomain"
 msgstr ""
 
-#: zerver/views/auth.py:822
+#: zerver/views/auth.py:823
 msgid "Subdomain required"
 msgstr ""
 
-#: zerver/views/auth.py:883
+#: zerver/views/auth.py:884
 msgid "GOOGLE_CLIENT_ID is not configured"
 msgstr ""
 

static/locale/zh_Hans/LC_MESSAGES/django.po

@@ -14,8 +14,8 @@ msgstr ""
 "Project-Id-Version: Zulip\n"
 "Report-Msgid-Bugs-To: \n"
 "POT-Creation-Date: 2018-11-03 00:32+0000\n"
-"PO-Revision-Date: 2018-11-03 00:04+0000\n"
-"Last-Translator: Tim Abbott <tabbott@kandralabs.com>\n"
+"PO-Revision-Date: 2018-11-08 07:30+0000\n"
+"Last-Translator: longjiang li <cqlilon@live.com>\n"
 "Language-Team: Chinese Simplified (http://www.transifex.com/zulip/zulip/language/zh-Hans/)\n"
 "MIME-Version: 1.0\n"
 "Content-Type: text/plain; charset=UTF-8\n"
@@ -318,7 +318,7 @@ msgstr "保存为草稿"
 
 #: templates/zerver/app/compose.html:20
 msgid "New message"
-msgstr ""
+msgstr "新消息"
 
 #: templates/zerver/app/compose.html:27 templates/zerver/app/compose.html:28
 msgid "New topic"
@@ -1380,7 +1380,7 @@ msgid ""
 "    If you did not request this change, please contact us immediately at\n"
 "    <a href=\"mailto:%(support_email)s\" style=\"color:hsl(164, 42%%, 47%%); text-decoration:underline\">%(support_email)s</a>.\n"
 "    "
-msgstr ""
+msgstr "\n如果您没有发送修改请求，请立即联系我们<a href=\"mailto:%(support_email)s\" style=\"color:hsl(164, 42%%, 47%%); text-decoration:underline\">%(support_email)s</a>"
 
 #: templates/zerver/emails/compiled/confirm_new_email.html:29
 #: templates/zerver/emails/compiled/confirm_registration.html:27
@@ -1450,7 +1450,7 @@ msgid ""
 "    <a href=\"mailto:%(support_email)s\" style=\"color:hsl(164, 42%%, 47%%); text-decoration:underline\">%(support_email)s</a>,\n"
 "    if you have any questions.\n"
 "    "
-msgstr ""
+msgstr "\n如果您有任何问题，都可以告诉我们<a href=\"mailto:%(support_email)s\" style=\"color:hsl(164, 42%%, 47%%); text-decoration:underline\">%(support_email)s</a>"
 
 #: templates/zerver/emails/compiled/find_team.html:9
 #: templates/zerver/emails/find_team.source.html:8
@@ -1511,7 +1511,7 @@ msgid ""
 "    Zulip works best when it's always open, so we suggest downloading\n"
 "    our <a href=\"https://zulipchat.com/apps\" style=\"color:hsl(164, 42%%, 47%%); text-decoration:underline\">desktop and mobile apps</a>.\n"
 "    "
-msgstr ""
+msgstr "\n为了使用Zulip时获得更好的体验，建议下载使用<a href=\"https://zulipchat.com/apps\" style=\"color:hsl(164, 42%%, 47%%); text-decoration:underline\">桌面端和移动端App</a>"
 
 #: templates/zerver/emails/compiled/followup_day1.html:19
 #: templates/zerver/emails/followup_day1.source.html:18
@@ -1528,7 +1528,7 @@ msgid ""
 "    Become a Zulip pro with a few\n"
 "    <a href=\"%(keyboard_shortcuts_link)s\" style=\"color:hsl(164, 42%%, 47%%); text-decoration:underline\">keyboard shortcuts</a>:\n"
 "    "
-msgstr ""
+msgstr "\n为了更方便的使用Zulip，了解一下<a href=\"%(keyboard_shortcuts_link)s\" style=\"color:hsl(164, 42%%, 47%%); text-decoration:underline\">快捷键</a>："
 
 #: templates/zerver/emails/compiled/followup_day1.html:34
 #: templates/zerver/emails/followup_day1.source.html:33
@@ -1555,7 +1555,7 @@ msgid ""
 "    Give our <a href=\"%(getting_started_link)s\" style=\"color:hsl(164, 42%%, 47%%); text-decoration:underline\">guide for new\n"
 "    %(user_role_group)s</a> a spin.\n"
 "    "
-msgstr ""
+msgstr "\n快速浏览<a href=\"%(getting_started_link)s\" style=\"color:hsl(164, 42%%, 47%%); text-decoration:underline\">%(user_role_group)s新手教程</a>"
 
 #: templates/zerver/emails/compiled/followup_day1.html:47
 msgid ""
@@ -1576,7 +1576,7 @@ msgid ""
 "    chat with us live on the\n"
 "    <a href=\"https://chat.zulip.org\" style=\"color:hsl(164, 42%%, 47%%); text-decoration:underline\">Zulip community server</a>!\n"
 "    "
-msgstr ""
+msgstr "\n例如：关注我们的<a href=\"https://twitter.com/zulip\" style=\"color:hsl(164, 42%%, 47%%); text-decoration:underline\">Twitter</a>，给我们的<a href=\"https://github.com/zulip/zulip\" style=\"color:hsl(164, 42%%, 47%%); text-decoration:underline\">GitHub</a>加星，或者在<a href=\"https://chat.zulip.org\" style=\"color:hsl(164, 42%%, 47%%); text-decoration:underline\">Zulip community server</a>和我们在线交流"
 
 #: templates/zerver/emails/compiled/invitation.html:12
 #, python-format
@@ -1601,7 +1601,7 @@ msgid ""
 "Feel free to give us a shout at <a href=\"mailto:%(support_email)s\" "
 "style=\"color:hsl(164, 42%%, 47%%); text-"
 "decoration:underline\">%(support_email)s</a>, if you have any questions."
-msgstr ""
+msgstr "如果您有任何问题，都可以告诉我们<a href=\"mailto:%(support_email)s\" style=\"color:hsl(164, 42%%, 47%%); text-decoration:underline\">%(support_email)s</a>"
 
 #: templates/zerver/emails/compiled/invitation.html:27
 #: templates/zerver/emails/compiled/invitation_reminder.html:23
@@ -1625,7 +1625,7 @@ msgid ""
 "href=\"mailto:%(referrer_email)s\" style=\"color:hsl(164, 42%%, 47%%); text-"
 "decoration:underline\">%(referrer_email)s</a>) wants you to join them on "
 "Zulip, a workplace chat tool that actually makes you more productive."
-msgstr ""
+msgstr "友情提示：%(referrer_name)s(<a href=\"mailto:%(referrer_email)s\" style=\"color:hsl(164, 42%%, 47%%); text-decoration:underline\">%(referrer_email)s</a>)邀请你加入他们的Zulip,一个能大大提高工作效率的交流工具"
 
 #: templates/zerver/emails/compiled/invitation_reminder.html:19
 #, python-format
@@ -1633,7 +1633,7 @@ msgid ""
 "We're here for you at <a href=\"mailto:%(support_email)s\" "
 "style=\"color:hsl(164, 42%%, 47%%); text-"
 "decoration:underline\">%(support_email)s</a> if you have any questions."
-msgstr ""
+msgstr "如果您有任何问题请联系我们<a href=\"mailto:%(support_email)s\" style=\"color:hsl(164, 42%%, 47%%); text-decoration:underline\">%(support_email)s</a>"
 
 #: templates/zerver/emails/compiled/notify_change_in_email.html:9
 #: templates/zerver/emails/notify_change_in_email.source.html:8
@@ -1651,7 +1651,7 @@ msgid ""
 "change, please contact us immediately at <a "
 "href=\"mailto:%(support_email)s\" style=\"color:hsl(164, 42%%, 47%%); text-"
 "decoration:underline\">%(support_email)s</a>."
-msgstr ""
+msgstr "您Zulip账户关联的电子邮件变更为<a href=\"%(new_email)s\" style=\"color:hsl(164, 42%%, 47%%); text-decoration:underline\">%(new_email)s</a>。如果不是您请求的变更请立即联系我们<a href=\"mailto:%(support_email)s\" style=\"color:hsl(164, 42%%, 47%%); text-decoration:underline\">%(support_email)s</a>"
 
 #: templates/zerver/emails/compiled/notify_change_in_email.html:14
 #: templates/zerver/emails/notify_change_in_email.source.html:13
@@ -2371,7 +2371,7 @@ msgstr "必须是组织管理员"
 
 #: zerver/decorator.py:142
 msgid "Must be a billing administrator or an organization administrator"
-msgstr ""
+msgstr "必须是账单管理员或者组织管理员"
 
 #: zerver/decorator.py:224
 msgid "Invalid subdomain for push notifications bouncer"
@@ -3011,7 +3011,7 @@ msgstr "名称中有无效字符！"
 
 #: zerver/lib/users.py:39
 msgid "Name is already in use!"
-msgstr ""
+msgstr "用户名已被占用"
 
 #: zerver/lib/users.py:44 zerver/views/users.py:283 zerver/views/users.py:455
 msgid "Bad name or username"
@@ -3736,7 +3736,7 @@ msgstr "无法停用唯一的社群管理员"
 
 #: zerver/views/users.py:97
 msgid "Guests cannot be organization administrators"
-msgstr ""
+msgstr "访客不能是组织管理员"
 
 #: zerver/views/users.py:101
 msgid "Cannot remove the only organization administrator"

static/locale/zh_Hans/translations.json

@@ -292,14 +292,14 @@
   "Mobile notifications": "移动端通知",
   "Mobile notifications always (even when online)": "移动端通知(在线)",
   "Mobile notifications when offline": "离线时移动端通知",
-  "Mobile push notifications are not configured on this server.": "",
+  "Mobile push notifications are not configured on this server.": "服务器未配置移动端推送通知",
   "More than 2 weeks ago": "超过2周",
   "Mute stream": "静音频道",
   "Mute the stream <b>__stream.name__</b>": "频道&quot;<b>__stream.name__</b>&quot;开启免打扰",
   "Mute the topic <b>__subject__</b>": "话题&quot;<b>__subject__</b>&quot;开启免打扰",
   "Mute the topic <b>__topic_name__</b>": "话题&quot;<b>__topic_name__</b>&quot;开启免打扰",
   "Mute topic": "静音主题",
-  "Muted streams don't show up in \\\"All messages\\\" or generate notifications unless you are mentioned.": "",
+  "Muted streams don't show up in \\\"All messages\\\" or generate notifications unless you are mentioned.": "开启免打扰频道不会出现在\\\"所有信息 \"\\中，也不会产生通知，除非您被@提醒。",
   "Muted topics": "已静音主题",
   "N": "N",
   "Name": "名称",
@@ -316,14 +316,14 @@
   "New conversation": "新对话",
   "New email": "新电子邮件",
   "New full name": "新全名",
-  "New members can only see messages sent after they join.": "",
-  "New members can view complete message history.": "",
+  "New members can only see messages sent after they join.": "新成员只能看到加入后发送的消息。",
+  "New members can view complete message history.": "新成员可以查看完整的消息历史。",
   "New password": "新密码",
-  "New password is too weak": "",
+  "New password is too weak": "新密码太弱",
   "New private message": "写私信",
   "New stream message": "写消息",
-  "New stream notifications:": "",
-  "New task": "",
+  "New stream notifications:": "新频道通知:",
+  "New task": "新任务",
   "New topic": "新话题",
   "New user notifications:": "新用户通知",
   "Next week": "下周",
@@ -334,7 +334,7 @@
   "No default streams match you current filter.": "没有默认频道可以匹配你当前的过滤器。",
   "No description.": "没有描述信息。",
   "No drafts.": "没有草稿",
-  "No invites match your current filter.": "",
+  "No invites match your current filter.": "没有邀请匹配当前过滤器。",
   "No more topics.": "没有更多话题。",
   "No restrictions": "无限制",
   "No users match your current filter.": "没有匹配到用户在你的筛选器中。",
@@ -346,25 +346,25 @@
   "Notifications stream changed!": "频道通知已更改！",
   "Notifications stream disabled!": "频道通知已禁用！",
   "Old password": "旧密码",
-  "On __last_active__": "",
-  "On __last_active_date__": "",
+  "On __last_active__": "在__last_active__",
+  "On __last_active_date__": "在__last_active_date__",
   "Only organization administrators can add bots to this organization": "只有组织管理员可以将机器人添加到该组织",
   "Only organization administrators can add custom emoji in this organization.": "只有社群管理员才能自定义表情在这个社群中。",
   "Only organization administrators can add generic bots": "只有组织管理员可以添加通用机器人",
   "Only organization administrators can edit these settings.": "只有社群管理员才能编辑这些设置。",
-  "Only organization administrators can post.": "",
-  "Only organization admins are allowed to post to this stream.": "",
+  "Only organization administrators can post.": "只有组织管理员才能发布。",
+  "Only organization admins are allowed to post to this stream.": "只有组织管理员可以发布到这个频道。",
   "Optional": "可选设置",
   "Organization": "社群",
-  "Organization administrators can change this in the organization settings.": "",
+  "Organization administrators can change this in the organization settings.": "组织管理员可以在组织设置中更改此设置。",
   "Organization avatar": "社群头像",
-  "Organization description": "",
+  "Organization description": "组织描述",
   "Organization name": "社群名称",
   "Organization permissions": "社群许可",
   "Organization profile": "社群资料",
   "Organization settings": "社区设置",
-  "Other notification settings": "",
-  "Other permissions": "",
+  "Other notification settings": "其他通知设置",
+  "Other permissions": "其他权限",
   "Outgoing webhook message format": "送出的webhook消息格式",
   "Owner": "所有者",
   "Password": "密码",
@@ -376,96 +376,96 @@
   "Pin stream to top of left sidebar": "钉住频道在左侧栏的顶部",
   "Please just upload one file.": "请上传一个文件",
   "Please re-enter your password to confirm your identity.": "请重新输入密码以确认你的身份。",
-  "Please specify a date or time": "",
+  "Please specify a date or time": "请注明日期或时间",
   "Please specify a stream": "请指定频道",
   "Please specify a topic": "请指定话题",
-  "Please specify at least one valid recipient": "",
+  "Please specify at least one valid recipient": "请至少指定一个可用的收信人",
   "Prevent users from changing their email address": "阻止用户更改邮件地址",
   "Prevent users from changing their name": "防止用户更改名称",
-  "Preview profile": "",
+  "Preview profile": "预览资料",
   "Private messages and @-mentions": "私信和@提醒",
-  "Profile": "",
-  "Profile field settings": "",
-  "Question": "",
+  "Profile": "资料",
+  "Profile field settings": "资料字段设置",
+  "Question": "问题",
   "Quote and reply": "引用并回复",
   "Reactivate": "启用",
   "Reactivate bot": "重启机器人",
   "Regular expression": "正则表达式",
-  "Remind me about this": "",
-  "Reminder not set!": "",
-  "Reminder set!": "",
+  "Remind me about this": "提醒我",
+  "Reminder not set!": "提醒没有设置!",
+  "Reminder set!": "提醒已设置!",
   "Remove": "移除",
   "Remove from default": "取消默认频道",
-  "Reply (r)": "",
+  "Reply (r)": "回复(r)",
   "Reply mentioning user": "回复提到用户",
   "Require topics in stream messages": "频道消息中所需的主题",
-  "Resend": "",
-  "Resend invitation to <span class=\"email\"></span>": "",
-  "Resend now": "",
-  "Resending encountered an error. Please reload and try again.": "",
+  "Resend": "重新发送",
+  "Resend invitation to <span class=\"email\"></span>": "重新发送邀请到<span class=\"email\"></span>",
+  "Resend now": "立即重新发送",
+  "Resending encountered an error. Please reload and try again.": "重发是发生错误。请刷新后再试",
   "Restore draft": "恢复草稿",
-  "Restrict email domains of new users?": "",
-  "Restrict posting to organization administrators": "",
-  "Restrict to a list of domains": "",
+  "Restrict email domains of new users?": "限制新用户的电子邮件域?",
+  "Restrict posting to organization administrators": "限制发布到组织管理员",
+  "Restrict to a list of domains": "限制到一个域列表",
   "Retry": "重试",
-  "Revoke": "",
-  "Revoke invitation to <span class=\"email\"></span>": "",
-  "Revoke now": "",
-  "Role": "",
+  "Revoke": "撤销",
+  "Revoke invitation to <span class=\"email\"></span>": "撤销发送给<span class=\"email\"></span>的邀请",
+  "Revoke now": "立即撤销",
+  "Role": "角色",
   "Save": "保存",
   "Save changes": "保存修改",
-  "Save failed": "",
-  "Saved": "",
-  "Saved. Please <a class='reload_link'>reload</a> for the change to take effect.": "",
-  "Saving": "",
+  "Save failed": "保存失败",
+  "Saved": "已保存",
+  "Saved. Please <a class='reload_link'>reload</a> for the change to take effect.": "已保存。请<a class='reload_link'>刷新</a>使更改生效",
+  "Saving": "保存中",
   "Search": "搜索",
   "Search operators": "搜索管理者",
   "Search results": "搜索结果",
   "Search subscribers": "搜索订阅者",
   "Search uploads...": "搜索已上传的文件",
   "See the rest of this message": "查看其余内容",
-  "Select date and time": "",
+  "Select date and time": "选择日期和时间",
   "Select default language": "选择默认语言",
-  "Send digest emails when I'm away": "",
-  "Send email notifications for new logins to my account": "",
-  "Send emails introducing Zulip to new users": "",
+  "Send digest emails when I'm away": "当我离线时发送摘要邮件",
+  "Send email notifications for new logins to my account": "我的帐户进行新的登录时发送电子邮件通知",
+  "Send emails introducing Zulip to new users": "向新用户发送介绍Zulip的电子邮件",
   "Send private message": "发送私有消息",
-  "Sent!": "",
+  "Sent!": "已发送!",
   "Settings": "设置",
-  "Setup": "",
-  "Setup two factor authentication": "",
-  "Show counts for starred messages": "",
+  "Setup": "设置",
+  "Setup two factor authentication": "设置双重认证",
+  "Show counts for starred messages": "显示星标消息的数量",
   "Show previews of linked websites": "显示链接网站的预览",
   "Show previews of uploaded and linked images": "显示上传文件链接的图像预览",
   "Show/change your API key": "显示/修改您的 API Key",
-  "Signup notifications stream changed!": "",
-  "Signup notifications stream disabled!": "",
+  "Signup notifications stream changed!": "频道注册通知已变更",
+  "Signup notifications stream disabled!": "频道注册通知已禁用",
   "Size": "大小",
   "Slack compatible": "高度兼容",
-  "Slack's outgoing webhooks": "",
+  "Slack's outgoing webhooks": "Slack发送的webhook",
   "Sorry, the file was too large.": "对不起，文件太大了。",
   "Star": "星标",
   "Stream": "频道",
   "Stream color": "频道颜色",
-  "Stream created recently": "",
+  "Stream created recently": "最近创建的频道",
   "Stream creation": "频道创建",
   "Stream description": "频道描述",
   "Stream description (optional)": "频道描述(可选)",
   "Stream membership": "频道用户",
   "Stream messages": "频道消息",
   "Stream name": "频道名称",
-  "Stream permissions": "",
+  "Stream permissions": "频道权限",
   "Stream settings": "频道设置",
-  "Stream successfully created!": "",
+  "Stream successfully created!": "频道创建成功",
   "Streams": "频道",
-  "Submit": "",
+  "Submit": "提交",
   "Subscribe": "订阅",
   "Subscribed": "已订阅",
-  "Subscribed successfully!": "",
-  "Subscriber count": "",
+  "Subscribed successfully!": "订阅成功",
+  "Subscriber count": "订阅者数量",
   "Subscribers": "订阅者",
-  "Task already exists": "",
-  "Text": "",
+  "Task already exists": "任务已经存在",
+  "Text": "文本",
   "The email body will become the Zulip message": "电子邮件正文将成为Zulip消息",
   "The email subject will become the Zulip topic": "电子邮件正文将成为Zulip话题",
   "The email will be forwarded to this stream": "邮件将会转发到这个频道中",
@@ -474,27 +474,27 @@
   "The stream description has been updated!": "频道描述信息已更新",
   "The stream has been renamed!": "频道重命名成功！",
   "The stream to which new stream notifications go to.": "新流通知发送到的频道。",
-  "The stream which new user signup notifications go to.": "",
+  "The stream which new user signup notifications go to.": "新用户注册通知频道",
   "Their password will be cleared from our systems, and any bots they maintain will be disabled.": "这些用户的密码会被从系统中清除，他们的机器人用户也会被关闭。",
-  "There are no messages to reply to.": "",
-  "These settings are explained in detail in the <a target=\"_blank\" href=\"/help/stream-permissions\">help center</a>.": "",
-  "This action is permanent and cannot be undone. All users will permanently lose access to their Zulip accounts.": "",
-  "This is a <span class=\"fa fa-globe\" aria-hidden=\"true\"></span> <b>public stream</b>. Anybody in your organization can join.": "",
-  "This is a <span class=\"fa fa-lock\" aria-hidden=\"true\"></span> <b>private stream</b>. Only people who have been invited can access its content, but any member of the stream can invite others.": "",
+  "There are no messages to reply to.": "没有消息可回复",
+  "These settings are explained in detail in the <a target=\"_blank\" href=\"/help/stream-permissions\">help center</a>.": "这些设置在<a target=\"_blank\" href=\"/help/stream-permissions\">帮助中心</a>中有详细说明。",
+  "This action is permanent and cannot be undone. All users will permanently lose access to their Zulip accounts.": "这项操作是永久且不可撤销的。所有用户将永久失去对Zulip账户的访问权限。",
+  "This is a <span class=\"fa fa-globe\" aria-hidden=\"true\"></span> <b>public stream</b>. Anybody in your organization can join.": "这是一个<span class=\"fa fa-globe\" aria-hidden=\"true\"></span><b>公共频道</b>。所有组织成员都可以加入",
+  "This is a <span class=\"fa fa-lock\" aria-hidden=\"true\"></span> <b>private stream</b>. Only people who have been invited can access its content, but any member of the stream can invite others.": "这是一个<span class=\"fa fa-lock\" aria-hidden=\"true\"></span><b>私有频道</b>。仅有邀请的用户可以对该频道进行访问，该频道的用户也可以邀请其它用户。",
   "This is a private stream": "这是一个私有频道",
   "This organization is configured to restrict editing of message content to __minutes_to_edit__ minutes after it is sent.": "这个社群组织已限制讯息发送间隔，请与 __minutes_to_edit__ 分钟后再发。",
-  "This stream is reserved for <strong>announcements</strong>. <br /> Are you sure you want to message all <strong>__count__</strong> people in this stream?": "",
+  "This stream is reserved for <strong>announcements</strong>. <br /> Are you sure you want to message all <strong>__count__</strong> people in this stream?": "此频道用于<strong>公告</strong>。<br />您确定向频道中所有<strong>__count__</strong>人发送消息吗？",
   "Time settings": "时间设置",
   "Time zone": "时区",
   "Time's up!": "时间到了！",
   "Today": "今日",
   "Toggle subscription": "触发订阅",
-  "Tomorrow": "",
+  "Tomorrow": "明天",
   "Topic": "话题",
   "Topic editing only": "只能主题编辑",
   "Try again": "再试一次",
-  "Two factor authentication": "",
-  "Type": "",
+  "Two factor authentication": "双重认证",
+  "Type": "类型",
   "URL format string": "URL格式",
   "Un-collapse": "展开",
   "Unable to upload that many files at once.": "无法一次上传这么多的文件。",
@@ -507,35 +507,35 @@
   "Unpin stream <b>__stream.name__</b> from top": "取消频道\"<b>__stream.name__</b>\"置顶",
   "Unstar": "取消星标",
   "Unsubscribe": "退订",
-  "Unsubscribed successfully!": "",
-  "Up to N minutes after posting": "",
-  "Up to __time_limit__ after posting": "",
+  "Unsubscribed successfully!": "退订成功",
+  "Up to N minutes after posting": "发布后N分钟",
+  "Up to __time_limit__ after posting": "发布后__time_limit__",
   "Update successful: Subdomains allowed for __domain__": "更新成功：允许新的域名于__domain__",
   "Update successful: Subdomains no longer allowed for __domain__": "更新成功：不在允许这个域名 __domain__",
-  "Updated settings!": "",
+  "Updated settings!": "更新设置",
   "Updated successfully!": "更新成功！",
   "Upload avatar": "上传头像",
   "Upload icon": "上传图标",
-  "Upload image or GIF": "",
+  "Upload image or GIF": "上传图片",
   "Upload new avatar": "上传一个新头像",
   "Upload new icon": "上传新图标",
   "Uploaded files": "已上传文件",
   "Uploading icon.": "图标上传中",
   "Uploading\u2026": "上传",
-  "User already subscribed.": "",
+  "User already subscribed.": "用户已经订阅",
   "User avatar": "用户头像",
-  "User group added!": "",
+  "User group added!": "用户组已添加",
   "User groups": "用户组",
   "User identity": "用户标识",
-  "User is already not subscribed.": "",
+  "User is already not subscribed.": "用户没有订阅",
   "User list on left sidebar in narrow windows": "窗口右侧变懒的用户列表",
-  "User role": "",
+  "User role": "用户角色",
   "User settings": "用户设置",
-  "User(s) invited successfully.": "",
+  "User(s) invited successfully.": "用户邀请成功",
   "Username": "用户名",
-  "Username (a-z, 0-9, and dashes only)": "",
-  "Users can edit the topic of any message": "",
-  "Video chat provider": "",
+  "Username (a-z, 0-9, and dashes only)": "用户名(字母、数字和下划线)",
+  "Users can edit the topic of any message": "用户可以编辑消息的主题",
+  "Video chat provider": "视频聊天提供者",
   "View edit history": "显示编辑历史 ",
   "View file": "显示文件",
   "View messages sent": "显示已发送消息",
@@ -543,58 +543,58 @@
   "View source": "显示源",
   "View source / Edit topic": "查看源 / 编辑主题",
   "View stream": "显示频道",
-  "View user profile": "",
-  "View your profile": "",
-  "Visual desktop notifications": "",
-  "Warning: <strong>__stream_name__</strong> is a private stream.": "",
-  "Who can add bots": "",
-  "Who can add custom emoji": "",
-  "Who can create streams": "",
+  "View user profile": "查看用户资料",
+  "View your profile": "查看我的资料",
+  "Visual desktop notifications": "可视桌面通知",
+  "Warning: <strong>__stream_name__</strong> is a private stream.": "警告:<strong>__stream_name__</strong>是私有频道",
+  "Who can add bots": "谁能添加机器人",
+  "Who can add custom emoji": "谁能添加自定义表情",
+  "Who can create streams": "谁能创建频道",
   "Working\u2026": "进行中",
   "Yes": "是",
   "Yes, delete this stream": "是的，删除该频道",
   "Yes, send": "是的，发送",
   "Yes, subscribe __count__ users!": "确定，订阅 __count__  用户！",
-  "Yes. Members and admins can send invitations.": "",
-  "Yes. Only admins can send invitations.": "",
+  "Yes. Members and admins can send invitations.": "是的，普通成员和管理员可以发送邀请",
+  "Yes. Only admins can send invitations.": "是的，只有管理员可以发送邀请",
   "Yesterday": "昨天",
   "You and __display_reply_to__": "您和__display_reply_to__",
   "You and __recipients__": "你和 __recipients__",
   "You are not currently subscribed to this stream.": "您目前未订阅此频道。",
   "You are not subscribed to stream __stream__": "你没有订阅__stream__频道",
   "You can send emails to Zulip! Just copy and use this address as an email recipient, and:": "您可以发送电子邮件给Zulip！ 只需复制并使用此地址作为电子邮件收件人，并且：",
-  "You cannot create a stream with no subscribers!": "",
+  "You cannot create a stream with no subscribers!": "创建频道时必须有订阅者",
   "You have no active bots.": "你没有可用的机器人。",
   "You have no inactive bots.": "你没有不可用的机器人。",
   "You have not muted any topics yet.": "你还没有任何静音的话题",
   "You have not uploaded any files.": "目前没有上传任何文件。",
   "You have nothing to send!": "消息不能为空！",
-  "You must be an organization administrator to create a stream without subscribing.": "",
+  "You must be an organization administrator to create a stream without subscribing.": "您必须是组织管理员才能在不订阅的情况下创建频道",
   "You need to be running Zephyr mirroring in order to send messages!": "您需要运行Zephyr镜像服务以便发送消息！",
   "You subscribed to stream __stream__": "你订阅了 __stream__ 频道",
   "You unsubscribed from stream __stream__": "你取消订阅了 __stream__ 频道",
-  "You're not subscribed to this stream. You will not be notified if other users reply to your message.": "",
+  "You're not subscribed to this stream. You will not be notified if other users reply to your message.": "您没有订阅这个频道。如果其他用户回复您的邮件，您将不会收到通知。",
   "Your API key:": "您的 API Key：",
   "Your account": "你的账户",
   "Your bots": "你的机器人",
-  "Your reminder note is empty!": "",
+  "Your reminder note is empty!": "您没有提醒事项",
   "[Condense this message]": "[收起消息]",
-  "[Configure]": "",
+  "[Configure]": "[配置]",
   "[Disable]": "[禁用]",
   "[More...]": "[更多...]",
-  "__hours__ hours ago": "",
+  "__hours__ hours ago": "__hours__小时以前",
   "__minutes__ min to edit": "__minutes__分钟内完成编辑",
-  "__minutes__ minutes ago": "",
+  "__minutes__ minutes ago": "__minutes__分钟以前",
   "__seconds__ sec to edit": "__seconds__秒内完成编辑",
   "__starred_status__ this message": "__starred_status__这个消息",
-  "__wildcard_mention_token__ (Notify stream)": "",
+  "__wildcard_mention_token__ (Notify stream)": "__wildcard_mention_token__ (通知频道)",
   "and": "来",
   "cookie": "cookie",
   "in 1 hour": "1小时内",
   "in 20 minutes": "20分钟内",
   "in 3 hours": "3小时内",
-  "leafy green vegetable": "",
-  "marketing": "",
+  "leafy green vegetable": "绿叶蔬菜",
+  "marketing": "销售",
   "more conversations": "更多会话",
   "more topics": "更多话题"
 }

templates/zerver/help/import-from-gitter.md

@@ -33,9 +33,10 @@ organization first.
 
 ### Import into a self-hosted Zulip server
 
-Because the import tool is very new, you will need to
-upgrade your Zulip server to the latest `master` branch,
-using [upgrade-zulip-from-git][upgrade-zulip-from-git].
+First
+[install a new Zulip server](https://zulip.readthedocs.io/en/stable/production/install.html),
+skipping "Step 3: Create a Zulip organization, and log in" (you'll
+create your Zulip organization via the data import tool instead).
 
 Log in to a shell on your Zulip server as the `zulip` user. To import with
 the most common configuration, run the following commands, replacing

templates/zerver/help/import-from-hipchat.md

@@ -65,6 +65,11 @@ organization first.
 
 ### Import into a self-hosted Zulip server
 
+First
+[install a new Zulip server](https://zulip.readthedocs.io/en/stable/production/install.html),
+skipping "Step 3: Create a Zulip organization, and log in" (you'll
+create your Zulip organization via the data import tool instead).
+
 Because the import tool is very new, you will need to
 upgrade your Zulip server to the latest `master` branch,
 using [upgrade-zulip-from-git][upgrade-zulip-from-git].

templates/zerver/help/import-from-slack.md

@@ -35,9 +35,10 @@ organization first.
 
 ### Import into a self-hosted Zulip server
 
-Because the import tool is very new, you will need to
-upgrade your Zulip server to the latest `master` branch,
-using [upgrade-zulip-from-git][upgrade-zulip-from-git].
+First
+[install a new Zulip server](https://zulip.readthedocs.io/en/stable/production/install.html),
+skipping "Step 3: Create a Zulip organization, and log in" (you'll
+create your Zulip organization via the data import tool instead).
 
 Log in to a shell on your Zulip server as the `zulip` user. To import with
 the most common configuration, run the following commands, replacing

tools/linter_lib/custom_check.py

@@ -770,6 +770,10 @@ def build_custom_checkers(by_lang):
          'include_only': set(['docs/']),
          'description': "Use relative links (../foo/bar.html) to other documents in docs/",
          },
+        {'pattern': "su zulip -c [^']",
+         'include_only': set(['docs/']),
+         'description': "Always quote arguments using `su zulip -c '` to avoid confusion about how su works.",
+         },
         {'pattern': r'\][(][^#h]',
          'include_only': set(['README.md', 'CONTRIBUTING.md']),
          'description': "Use absolute links from docs served by GitHub",

version.py

@@ -1,4 +1,4 @@
-ZULIP_VERSION = "1.9.0"
+ZULIP_VERSION = "1.9.2"
 
 # Bump the minor PROVISION_VERSION to indicate that folks should provision
 # only when going from an old version of the code to a newer version. Bump

zerver/data_import/gitter.py

@@ -217,8 +217,13 @@ def get_usermentions(message: Dict[str, Any], user_map: Dict[str, int],
         for mention in message['mentions']:
             if mention.get('userId') in user_map:
                 gitter_mention = '@%s' % (mention['screenName'])
-                zulip_mention = ('@**%s**' %
-                                 (user_short_name_to_full_name[mention['screenName']]))
+                if mention['screenName'] not in user_short_name_to_full_name:
+                    logging.info("Mentioned user %s never sent any messages, so has no full name data" %
+                                 mention['screenName'])
+                    full_name = mention['screenName']
+                else:
+                    full_name = user_short_name_to_full_name[mention['screenName']]
+                zulip_mention = ('@**%s**' % (full_name,))
                 message['text'] = message['text'].replace(gitter_mention, zulip_mention)
 
                 mentioned_user_ids.append(user_map[mention['userId']])

zerver/data_import/hipchat.py

@@ -313,16 +313,31 @@ def write_emoticon_data(realm_id: int,
 
     fn = 'emoticons.json'
     data_file = os.path.join(data_dir, fn)
+    if not os.path.exists(data_file):
+        logging.warning("HipChat export does not contain emoticons.json.")
+        logging.warning("As a result, custom emoji cannot be imported.")
+        return []
+
     with open(data_file) as f:
         data = ujson.load(f)
 
-    flat_data = [
-        dict(
-            path=d['Emoticon']['path'],
-            name=d['Emoticon']['shortcut'],
-        )
-        for d in data
-    ]
+    if isinstance(data, dict) and 'Emoticons' in data:
+        # Handle the hc-migrate export format for emoticons.json.
+        flat_data = [
+            dict(
+                path=d['path'],
+                name=d['shortcut'],
+            )
+            for d in data['Emoticons']
+        ]
+    else:
+        flat_data = [
+            dict(
+                path=d['Emoticon']['path'],
+                name=d['Emoticon']['shortcut'],
+            )
+            for d in data
+        ]
 
     emoji_folder = os.path.join(output_dir, 'emoji')
     os.makedirs(emoji_folder, exist_ok=True)
@@ -506,7 +521,7 @@ def process_message_file(realm_id: int,
             )
 
             if is_pm_data:
-                if sender_id != fn_id:
+                if int(sender_id) != int(fn_id):
                     # PMs are in multiple places in the Hipchat export,
                     # and we only use the copy from the sender
                     return None

zerver/data_import/import_util.py

@@ -3,9 +3,10 @@ import requests
 import shutil
 import logging
 import os
+import traceback
 import ujson
 
-from typing import List, Dict, Any, Optional, Set, Callable
+from typing import List, Dict, Any, Optional, Set, Callable, Iterable, Tuple, TypeVar
 from django.forms.models import model_to_dict
 
 from zerver.models import Realm, RealmEmoji, Subscription, Recipient, \
@@ -13,7 +14,7 @@ from zerver.models import Realm, RealmEmoji, Subscription, Recipient, \
 from zerver.data_import.sequencer import NEXT_ID
 from zerver.lib.actions import STREAM_ASSIGNMENT_COLORS as stream_colors
 from zerver.lib.avatar_hash import user_avatar_path_from_ids
-from zerver.lib.parallel import run_parallel
+from zerver.lib.parallel import run_parallel, JobData
 
 # stubs
 ZerverFieldsT = Dict[str, Any]
@@ -263,9 +264,15 @@ def build_usermessages(zerver_usermessage: List[ZerverFieldsT],
                        subscriber_map: Dict[int, Set[int]],
                        recipient_id: int,
                        mentioned_user_ids: List[int],
-                       message_id: int) -> None:
+                       message_id: int,
+                       long_term_idle: Optional[Set[int]]=None) -> Tuple[int, int]:
     user_ids = subscriber_map.get(recipient_id, set())
 
+    if long_term_idle is None:
+        long_term_idle = set()
+
+    user_messages_created = 0
+    user_messages_skipped = 0
     if user_ids:
         for user_id in sorted(user_ids):
             is_mentioned = user_id in mentioned_user_ids
@@ -274,6 +281,12 @@ def build_usermessages(zerver_usermessage: List[ZerverFieldsT],
             # It's possible we don't even get PMs from them.
             is_private = False
 
+            if not is_mentioned and not is_private and user_id in long_term_idle:
+                # these users are long-term idle
+                user_messages_skipped += 1
+                continue
+            user_messages_created += 1
+
             usermessage = build_user_message(
                 user_id=user_id,
                 message_id=message_id,
@@ -282,6 +295,7 @@ def build_usermessages(zerver_usermessage: List[ZerverFieldsT],
             )
 
             zerver_usermessage.append(usermessage)
+    return (user_messages_created, user_messages_skipped)
 
 def build_user_message(user_id: int,
                        message_id: int,
@@ -388,17 +402,16 @@ def process_avatars(avatar_list: List[ZerverFieldsT], avatar_dir: str, realm_id:
     downloaded.  For simpler conversions see write_avatar_png.
     """
 
-    def get_avatar(avatar_upload_list: List[str]) -> int:
-        avatar_url = avatar_upload_list[0]
+    def get_avatar(avatar_upload_item: List[str]) -> None:
+        avatar_url = avatar_upload_item[0]
 
-        image_path = os.path.join(avatar_dir, avatar_original_list[1])
-        original_image_path = os.path.join(avatar_dir, avatar_original_list[2])
+        image_path = os.path.join(avatar_dir, avatar_upload_item[1])
+        original_image_path = os.path.join(avatar_dir, avatar_upload_item[2])
 
         response = requests.get(avatar_url + size_url_suffix, stream=True)
         with open(image_path, 'wb') as image_file:
             shutil.copyfileobj(response.raw, image_file)
         shutil.copy(image_path, original_image_path)
-        return 0
 
     logging.info('######### GETTING AVATARS #########\n')
     logging.info('DOWNLOADING AVATARS .......\n')
@@ -425,7 +438,7 @@ def process_avatars(avatar_list: List[ZerverFieldsT], avatar_dir: str, realm_id:
 
     # Run downloads parallely
     output = []
-    for (status, job) in run_parallel(get_avatar, avatar_upload_list, threads=threads):
+    for (status, job) in run_parallel_wrapper(get_avatar, avatar_upload_list, threads=threads):
         output.append(job)
 
     logging.info('######### GETTING AVATARS FINISHED #########\n')
@@ -458,10 +471,32 @@ def write_avatar_png(avatar_folder: str,
         s3_path=image_path,
         realm_id=realm_id,
         user_profile_id=user_id,
+        # We only write the .original file; ask the importer to do the thumbnailing.
+        importer_should_thumbnail=True,
     )
 
     return metadata
 
+ListJobData = TypeVar('ListJobData')
+def run_parallel_wrapper(f: Callable[[ListJobData], None], full_items: List[ListJobData],
+                         threads: int=6) -> Iterable[Tuple[int, List[ListJobData]]]:
+    logging.info("Distributing %s items across %s threads" % (len(full_items), threads))
+
+    def wrapping_function(items: List[ListJobData]) -> int:
+        count = 0
+        for item in items:
+            try:
+                f(item)
+            except Exception:
+                logging.info("Error processing item: %s" % (item,))
+                traceback.print_exc()
+            count += 1
+            if count % 1000 == 0:
+                logging.info("A download thread finished %s items" % (count,))
+        return 0
+    job_lists = [full_items[i::threads] for i in range(threads)]  # type: List[List[ListJobData]]
+    return run_parallel(wrapping_function, job_lists, threads=threads)
+
 def process_uploads(upload_list: List[ZerverFieldsT], upload_dir: str,
                     threads: int) -> List[ZerverFieldsT]:
     """
@@ -471,7 +506,7 @@ def process_uploads(upload_list: List[ZerverFieldsT], upload_dir: str,
     1. upload_list: List of uploads to be mapped in uploads records.json file
     2. upload_dir: Folder where the downloaded uploads are saved
     """
-    def get_uploads(upload: List[str]) -> int:
+    def get_uploads(upload: List[str]) -> None:
         upload_url = upload[0]
         upload_path = upload[1]
         upload_path = os.path.join(upload_dir, upload_path)
@@ -480,7 +515,6 @@ def process_uploads(upload_list: List[ZerverFieldsT], upload_dir: str,
         os.makedirs(os.path.dirname(upload_path), exist_ok=True)
         with open(upload_path, 'wb') as upload_file:
             shutil.copyfileobj(response.raw, upload_file)
-        return 0
 
     logging.info('######### GETTING ATTACHMENTS #########\n')
     logging.info('DOWNLOADING ATTACHMENTS .......\n')
@@ -493,7 +527,7 @@ def process_uploads(upload_list: List[ZerverFieldsT], upload_dir: str,
 
     # Run downloads parallely
     output = []
-    for (status, job) in run_parallel(get_uploads, upload_url_list, threads=threads):
+    for (status, job) in run_parallel_wrapper(get_uploads, upload_url_list, threads=threads):
         output.append(job)
 
     logging.info('######### GETTING ATTACHMENTS FINISHED #########\n')
@@ -522,7 +556,7 @@ def process_emojis(zerver_realmemoji: List[ZerverFieldsT], emoji_dir: str,
     2. emoji_dir: Folder where the downloaded emojis are saved
     3. emoji_url_map: Maps emoji name to its url
     """
-    def get_emojis(upload: List[str]) -> int:
+    def get_emojis(upload: List[str]) -> None:
         emoji_url = upload[0]
         emoji_path = upload[1]
         upload_emoji_path = os.path.join(emoji_dir, emoji_path)
@@ -531,7 +565,6 @@ def process_emojis(zerver_realmemoji: List[ZerverFieldsT], emoji_dir: str,
         os.makedirs(os.path.dirname(upload_emoji_path), exist_ok=True)
         with open(upload_emoji_path, 'wb') as emoji_file:
             shutil.copyfileobj(response.raw, emoji_file)
-        return 0
 
     emoji_records = []
     upload_emoji_list = []
@@ -555,7 +588,7 @@ def process_emojis(zerver_realmemoji: List[ZerverFieldsT], emoji_dir: str,
 
     # Run downloads parallely
     output = []
-    for (status, job) in run_parallel(get_emojis, upload_emoji_list, threads=threads):
+    for (status, job) in run_parallel_wrapper(get_emojis, upload_emoji_list, threads=threads):
         output.append(job)
 
     logging.info('######### GETTING EMOJIS FINISHED #########\n')

zerver/data_import/slack.py

@@ -10,11 +10,13 @@ import logging
 import random
 import requests
 
+from collections import defaultdict
+
 from django.conf import settings
 from django.db import connection
 from django.utils.timezone import now as timezone_now
 from django.forms.models import model_to_dict
-from typing import Any, Dict, List, Optional, Tuple, Set
+from typing import Any, Dict, List, Optional, Tuple, Set, Iterator
 from zerver.forms import check_subdomain_available
 from zerver.models import Reaction, RealmEmoji, Realm, UserProfile, Recipient, \
     CustomProfileField, CustomProfileFieldValue
@@ -263,6 +265,9 @@ def build_customprofilefields_values(custom_field_map: ZerverFieldsT, fields: Ze
                                      user_id: int, custom_field_id: int,
                                      custom_field_values: List[ZerverFieldsT]) -> int:
     for field, value in fields.items():
+        if value['value'] == "":
+            # Skip writing entries for fields with an empty value
+            continue
         custom_field_value = CustomProfileFieldValue(
             id=custom_field_id,
             value=value['value'])
@@ -296,7 +301,7 @@ def get_user_email(user: ZerverFieldsT, domain_name: str) -> str:
         else:
             raise AssertionError("Could not identify bot type")
         return slack_bot_name.replace("Bot", "").replace(" ", "") + "-bot@%s" % (domain_name,)
-    if get_user_full_name(user) == "slackbot":
+    if get_user_full_name(user).lower() == "slackbot":
         return "imported-slackbot-bot@%s" % (domain_name,)
     raise AssertionError("Could not find email address for Slack user %s" % (user,))
 
@@ -434,9 +439,61 @@ def get_subscription(channel_members: List[str], zerver_subscription: List[Zerve
         subscription_id += 1
     return subscription_id
 
+def process_long_term_idle_users(slack_data_dir: str, users: List[ZerverFieldsT],
+                                 added_users: AddedUsersT, added_channels: AddedChannelsT,
+                                 zerver_userprofile: List[ZerverFieldsT]) -> Set[int]:
+    """Algorithmically, we treat users who have sent at least 10 messages
+    or have sent a message within the last 60 days as active.
+    Everyone else is treated as long-term idle, which means they will
+    have a slighly slower first page load when coming back to
+    Zulip.
+    """
+    all_messages = get_messages_iterator(slack_data_dir, added_channels)
+
+    sender_counts = defaultdict(int)  # type: Dict[str, int]
+    recent_senders = set()  # type: Set[str]
+    NOW = float(timezone_now().timestamp())
+    for message in all_messages:
+        timestamp = float(message['ts'])
+        slack_user_id = get_message_sending_user(message)
+        if not slack_user_id:
+            # Ignore messages without user names
+            continue
+
+        if slack_user_id in recent_senders:
+            continue
+
+        if NOW - timestamp < 60:
+            recent_senders.add(slack_user_id)
+
+        sender_counts[slack_user_id] += 1
+    for (slack_sender_id, count) in sender_counts.items():
+        if count > 10:
+            recent_senders.add(slack_sender_id)
+
+    long_term_idle = set()
+
+    for slack_user in users:
+        if slack_user["id"] in recent_senders:
+            continue
+        zulip_user_id = added_users[slack_user['id']]
+        long_term_idle.add(zulip_user_id)
+
+    # Record long-term idle status in zerver_userprofile
+    for user_profile_row in zerver_userprofile:
+        if user_profile_row['id'] in long_term_idle:
+            user_profile_row['long_term_idle'] = True
+            # Setting last_active_message_id to 1 means the user, if
+            # imported, will get the full message history for the
+            # streams they were on.
+            user_profile_row['last_active_message_id'] = 1
+
+    return long_term_idle
+
 def convert_slack_workspace_messages(slack_data_dir: str, users: List[ZerverFieldsT], realm_id: int,
                                      added_users: AddedUsersT, added_recipient: AddedRecipientsT,
                                      added_channels: AddedChannelsT, realm: ZerverFieldsT,
+                                     zerver_userprofile: List[ZerverFieldsT],
                                      zerver_realmemoji: List[ZerverFieldsT], domain_name: str,
                                      output_dir: str,
                                      chunk_size: int=MESSAGE_BATCH_CHUNK_SIZE) -> Tuple[List[ZerverFieldsT],
@@ -448,12 +505,12 @@ def convert_slack_workspace_messages(slack_data_dir: str, users: List[ZerverFiel
     2. uploads, which is a list of uploads to be mapped in uploads records.json
     3. attachment, which is a list of the attachments
     """
-    all_messages = get_all_messages(slack_data_dir, added_channels)
 
-    # we sort the messages according to the timestamp to show messages with
-    # the proper date order
-    all_messages = sorted(all_messages, key=lambda message: message['ts'])
+    long_term_idle = process_long_term_idle_users(slack_data_dir, users, added_users,
+                                                  added_channels, zerver_userprofile)
 
+    # Now, we actually import the messages.
+    all_messages = get_messages_iterator(slack_data_dir, added_channels)
     logging.info('######### IMPORTING MESSAGES STARTED #########\n')
 
     total_reactions = []  # type: List[ZerverFieldsT]
@@ -461,8 +518,6 @@ def convert_slack_workspace_messages(slack_data_dir: str, users: List[ZerverFiel
     total_uploads = []  # type: List[ZerverFieldsT]
 
     # The messages are stored in batches
-    low_index = 0
-    upper_index = low_index + chunk_size
     dump_file_id = 1
 
     subscriber_map = make_subscriber_map(
@@ -470,14 +525,21 @@ def convert_slack_workspace_messages(slack_data_dir: str, users: List[ZerverFiel
     )
 
     while True:
-        message_data = all_messages[low_index:upper_index]
+        message_data = []
+        _counter = 0
+        for msg in all_messages:
+            _counter += 1
+            message_data.append(msg)
+            if _counter == chunk_size:
+                break
         if len(message_data) == 0:
             break
+
         zerver_message, zerver_usermessage, attachment, uploads, reactions = \
             channel_message_to_zerver_message(
                 realm_id, users, added_users, added_recipient, message_data,
                 zerver_realmemoji, subscriber_map, added_channels,
-                domain_name)
+                domain_name, long_term_idle)
 
         message_json = dict(
             zerver_message=zerver_message,
@@ -491,26 +553,39 @@ def convert_slack_workspace_messages(slack_data_dir: str, users: List[ZerverFiel
         total_attachments += attachment
         total_uploads += uploads
 
-        low_index = upper_index
-        upper_index = chunk_size + low_index
         dump_file_id += 1
 
     logging.info('######### IMPORTING MESSAGES FINISHED #########\n')
     return total_reactions, total_uploads, total_attachments
 
-def get_all_messages(slack_data_dir: str, added_channels: AddedChannelsT) -> List[ZerverFieldsT]:
-    all_messages = []  # type: List[ZerverFieldsT]
+def get_messages_iterator(slack_data_dir: str, added_channels: AddedChannelsT) -> Iterator[ZerverFieldsT]:
+    """This function is an iterator that returns all the messages across
+       all Slack channels, in order by timestamp.  It's important to
+       not read all the messages into memory at once, because for
+       large imports that can OOM kill."""
+    all_json_names = defaultdict(list)  # type: Dict[str, List[str]]
     for channel_name in added_channels.keys():
         channel_dir = os.path.join(slack_data_dir, channel_name)
         json_names = os.listdir(channel_dir)
         for json_name in json_names:
+            all_json_names[json_name].append(channel_dir)
+
+    # Sort json_name by date
+    for json_name in sorted(all_json_names.keys()):
+        messages_for_one_day = []  # type: List[ZerverFieldsT]
+        for channel_dir in all_json_names[json_name]:
             message_dir = os.path.join(channel_dir, json_name)
             messages = get_data_file(message_dir)
+            channel_name = os.path.basename(channel_dir)
             for message in messages:
                 # To give every message the channel information
                 message['channel_name'] = channel_name
-            all_messages += messages
-    return all_messages
+            messages_for_one_day += messages
+
+        # we sort the messages according to the timestamp to show messages with
+        # the proper date order
+        for message in sorted(messages_for_one_day, key=lambda m: m['ts']):
+            yield message
 
 def channel_message_to_zerver_message(realm_id: int,
                                       users: List[ZerverFieldsT],
@@ -520,11 +595,12 @@ def channel_message_to_zerver_message(realm_id: int,
                                       zerver_realmemoji: List[ZerverFieldsT],
                                       subscriber_map: Dict[int, Set[int]],
                                       added_channels: AddedChannelsT,
-                                      domain_name: str) -> Tuple[List[ZerverFieldsT],
-                                                                 List[ZerverFieldsT],
-                                                                 List[ZerverFieldsT],
-                                                                 List[ZerverFieldsT],
-                                                                 List[ZerverFieldsT]]:
+                                      domain_name: str,
+                                      long_term_idle: Set[int]) -> Tuple[List[ZerverFieldsT],
+                                                                         List[ZerverFieldsT],
+                                                                         List[ZerverFieldsT],
+                                                                         List[ZerverFieldsT],
+                                                                         List[ZerverFieldsT]]:
     """
     Returns:
     1. zerver_message, which is a list of the messages
@@ -543,6 +619,8 @@ def channel_message_to_zerver_message(realm_id: int,
     with open(NAME_TO_CODEPOINT_PATH) as fp:
         name_to_codepoint = ujson.load(fp)
 
+    total_user_messages = 0
+    total_skipped_user_messages = 0
     for message in all_messages:
         user = get_message_sending_user(message)
         if not user:
@@ -620,14 +698,19 @@ def channel_message_to_zerver_message(realm_id: int,
         zerver_message.append(zulip_message)
 
         # construct usermessages
-        build_usermessages(
+        (num_created, num_skipped) = build_usermessages(
             zerver_usermessage=zerver_usermessage,
             subscriber_map=subscriber_map,
             recipient_id=recipient_id,
             mentioned_user_ids=mentioned_user_ids,
             message_id=message_id,
+            long_term_idle=long_term_idle,
         )
+        total_user_messages += num_created
+        total_skipped_user_messages += num_skipped
 
+    logging.debug("Created %s UserMessages; deferred %s due to long-term idle" % (
+        total_user_messages, total_skipped_user_messages))
     return zerver_message, zerver_usermessage, zerver_attachment, uploads_list, \
         reaction_list
 
@@ -802,7 +885,7 @@ def do_convert_data(slack_zip_file: str, output_dir: str, token: str, threads: i
 
     reactions, uploads_list, zerver_attachment = convert_slack_workspace_messages(
         slack_data_dir, user_list, realm_id, added_users, added_recipient, added_channels,
-        realm, realm['zerver_realmemoji'], domain_name, output_dir)
+        realm, realm['zerver_userprofile'], realm['zerver_realmemoji'], domain_name, output_dir)
 
     # Move zerver_reactions to realm.json file
     realm['zerver_reaction'] = reactions

zerver/lib/export.py

@@ -1287,8 +1287,6 @@ def do_export_realm(realm: Realm, output_dir: Path, threads: int,
     if not settings.TEST_SUITE:
         assert threads >= 1
 
-    assert os.path.exists("./manage.py")
-
     realm_config = get_realm_config()
 
     create_soft_link(source=output_dir, in_progress=True)

zerver/lib/import_realm.py

@@ -599,7 +599,8 @@ def import_uploads_s3(bucket_name: str, import_dir: Path, processing_avatars: bo
             user_profile = get_user_profile_by_id(user_profile_id)
             key.set_metadata("user_profile_id", str(user_profile.id))
 
-        key.set_metadata("orig_last_modified", record['last_modified'])
+        if 'last_modified' in record:
+            key.set_metadata("orig_last_modified", record['last_modified'])
         key.set_metadata("realm_id", str(record['realm_id']))
 
         # Zulip exports will always have a content-type, but third-party exports might not.
@@ -620,6 +621,8 @@ def import_uploads_s3(bucket_name: str, import_dir: Path, processing_avatars: bo
             if record['s3_path'].endswith('.original'):
                 user_profile = get_user_profile_by_id(record['user_profile_id'])
                 upload_backend.ensure_medium_avatar_image(user_profile=user_profile)
+                if record.get("importer_should_thumbnail"):
+                    upload_backend.ensure_basic_avatar_image(user_profile=user_profile)
 
 def import_uploads(import_dir: Path, processing_avatars: bool=False,
                    processing_emojis: bool=False) -> None:
@@ -722,6 +725,10 @@ def do_import_realm(import_dir: Path, subdomain: str) -> Realm:
     # Remap the user IDs for notification_bot and friends to their
     # appropriate IDs on this server
     for item in data['zerver_userprofile_crossrealm']:
+        if item['email'].startswith("emailgateway@"):
+            # The email gateway bot's email is customized to a
+            # different domain on some servers.
+            item['email'] = settings.EMAIL_GATEWAY_BOT
         logging.info("Adding to ID map: %s %s" % (item['id'], get_system_bot(item['email']).id))
         new_user_id = get_system_bot(item['email']).id
         update_id_map(table='user_profile', old_id=item['id'], new_id=new_user_id)
@@ -902,6 +909,29 @@ def do_import_realm(import_dir: Path, subdomain: str) -> Realm:
     update_model_ids(Reaction, data, 'reaction')
     bulk_import_model(data, Reaction)
 
+    for user_profile in UserProfile.objects.filter(is_bot=False, realm=realm):
+        # Since we now unconditionally renumbers message IDs, we need
+        # to reset the user's pointer to what will be a valid value.
+        #
+        # For zulip->zulip imports, we could do something clever, but
+        # it should always be safe to reset to first unread message.
+        #
+        # Longer-term, the plan is to eliminate pointer as a concept.
+        first_unread_message = UserMessage.objects.filter(user_profile=user_profile).extra(
+            where=[UserMessage.where_unread()]
+        ).first()
+        if first_unread_message is not None:
+            user_profile.pointer = first_unread_message.message_id
+        else:
+            last_message = UserMessage.objects.filter(user_profile=user_profile).last()
+            if last_message is not None:
+                user_profile.pointer = last_message.message_id
+            else:
+                # -1 is the guard value for new user accounts with no messages.
+                user_profile.pointer = -1
+
+        user_profile.save(update_fields=["pointer"])
+
     # Do attachments AFTER message data is loaded.
     # TODO: de-dup how we read these json files.
     fn = os.path.join(import_dir, "attachment.json")

zerver/lib/push_notifications.py

@@ -114,8 +114,8 @@ def send_apple_push_notification(user_id: int, devices: List[DeviceToken],
 
     client = get_apns_client()  # type: APNsClient
     if client is None:
-        logging.warning("APNs: Dropping a notification because nothing configured.  "
-                        "Set PUSH_NOTIFICATION_BOUNCER_URL (or APNS_CERT_FILE).")
+        logging.debug("APNs: Dropping a notification because nothing configured.  "
+                      "Set PUSH_NOTIFICATION_BOUNCER_URL (or APNS_CERT_FILE).")
         return
 
     if remote:
@@ -186,8 +186,8 @@ def send_android_push_notification_to_user(user_profile: UserProfile, data: Dict
 def send_android_push_notification(devices: List[DeviceToken], data: Dict[str, Any],
                                    remote: bool=False) -> None:
     if not gcm:
-        logging.warning("Skipping sending a GCM push notification since "
-                        "PUSH_NOTIFICATION_BOUNCER_URL and ANDROID_GCM_API_KEY are both unset")
+        logging.debug("Skipping sending a GCM push notification since "
+                      "PUSH_NOTIFICATION_BOUNCER_URL and ANDROID_GCM_API_KEY are both unset")
         return
     reg_ids = [device.token for device in devices]
 
@@ -429,6 +429,12 @@ def push_notifications_enabled() -> bool:
         return True
     return False
 
+def initialize_push_notifications() -> None:
+    if not push_notifications_enabled():
+        logging.warning("Mobile push notifications are not configured.\n  "
+                        "See https://zulip.readthedocs.io/en/latest/"
+                        "production/mobile-push-notifications.html")
+
 def get_gcm_alert(message: Message) -> str:
     """
     Determine what alert string to display based on the missed messages.
@@ -637,7 +643,7 @@ def handle_push_notification(user_profile_id: int, missed_message: Dict[str, Any
     user_profile = get_user_profile_by_id(user_profile_id)
     (message, user_message) = access_message(user_profile, missed_message['message_id'])
     if user_message is not None:
-        # If ther user has read the message already, don't push-notify.
+        # If the user has read the message already, don't push-notify.
         #
         # TODO: It feels like this is already handled when things are
         # put in the queue; maybe we should centralize this logic with

zerver/lib/upload.py

@@ -182,6 +182,9 @@ class ZulipUploadBackend:
     def ensure_medium_avatar_image(self, user_profile: UserProfile) -> None:
         raise NotImplementedError()
 
+    def ensure_basic_avatar_image(self, user_profile: UserProfile) -> None:
+        raise NotImplementedError()
+
     def upload_realm_icon_image(self, icon_file: File, user_profile: UserProfile) -> None:
         raise NotImplementedError()
 
@@ -431,7 +434,7 @@ class S3UploadBackend(ZulipUploadBackend):
         bucket_name = settings.S3_AVATAR_BUCKET
         conn = S3Connection(settings.S3_KEY, settings.S3_SECRET_KEY)
         bucket = get_bucket(conn, bucket_name)
-        key = bucket.get_key(file_path)
+        key = bucket.get_key(file_path + ".original")
         image_data = key.get_contents_as_string()
 
         resized_medium = resize_avatar(image_data, MEDIUM_AVATAR_SIZE)  # type: ignore # image_data is `bytes`, boto subs are wrong
@@ -443,6 +446,27 @@ class S3UploadBackend(ZulipUploadBackend):
             resized_medium
         )
 
+    def ensure_basic_avatar_image(self, user_profile: UserProfile) -> None:  # nocoverage
+        # TODO: Refactor this to share code with ensure_medium_avatar_image
+        file_path = user_avatar_path(user_profile)
+        # Also TODO: Migrate to user_avatar_path(user_profile) + ".png".
+        s3_file_name = file_path
+
+        bucket_name = settings.S3_AVATAR_BUCKET
+        conn = S3Connection(settings.S3_KEY, settings.S3_SECRET_KEY)
+        bucket = get_bucket(conn, bucket_name)
+        key = bucket.get_key(file_path + ".original")
+        image_data = key.get_contents_as_string()
+
+        resized_avatar = resize_avatar(image_data)  # type: ignore # image_data is `bytes`, boto subs are wrong
+        upload_image_to_s3(
+            bucket_name,
+            s3_file_name,
+            "image/png",
+            user_profile,
+            resized_avatar
+        )
+
     def upload_emoji_image(self, emoji_file: File, emoji_file_name: str,
                            user_profile: UserProfile) -> None:
         content_type = guess_type(emoji_file.name)[0]
@@ -589,6 +613,19 @@ class LocalUploadBackend(ZulipUploadBackend):
         resized_medium = resize_avatar(image_data, MEDIUM_AVATAR_SIZE)
         write_local_file('avatars', file_path + '-medium.png', resized_medium)
 
+    def ensure_basic_avatar_image(self, user_profile: UserProfile) -> None:  # nocoverage
+        # TODO: Refactor this to share code with ensure_medium_avatar_image
+        file_path = user_avatar_path(user_profile)
+
+        output_path = os.path.join(settings.LOCAL_UPLOADS_DIR, "avatars", file_path + ".png")
+        if os.path.isfile(output_path):
+            return
+
+        image_path = os.path.join(settings.LOCAL_UPLOADS_DIR, "avatars", file_path + ".original")
+        image_data = open(image_path, "rb").read()
+        resized_avatar = resize_avatar(image_data)
+        write_local_file('avatars', file_path + '.png', resized_avatar)
+
     def upload_emoji_image(self, emoji_file: File, emoji_file_name: str,
                            user_profile: UserProfile) -> None:
         emoji_path = RealmEmoji.PATH_ID_TEMPLATE.format(

zerver/management/commands/show_admins.py

@@ -3,26 +3,20 @@ import sys
 from argparse import ArgumentParser
 from typing import Any
 
-from django.core.management.base import BaseCommand
+from django.core.management.base import CommandError
 
-from zerver.models import Realm, get_realm
+from zerver.lib.management import ZulipBaseCommand
+from zerver.models import get_realm
 
-class Command(BaseCommand):
+class Command(ZulipBaseCommand):
     help = """Show the admins in a realm."""
 
     def add_arguments(self, parser: ArgumentParser) -> None:
-        parser.add_argument('realm', metavar='<realm>', type=str,
-                            help="realm to show admins for")
-
-    def handle(self, *args: Any, **options: str) -> None:
-        realm_name = options['realm']
-
-        try:
-            realm = get_realm(realm_name)
-        except Realm.DoesNotExist:
-            print('There is no realm called %s.' % (realm_name,))
-            sys.exit(1)
+        self.add_realm_args(parser, required=True)
 
+    def handle(self, *args: Any, **options: Any) -> None:
+        realm = self.get_realm(options)
+        assert realm is not None  # True because of required=True above
         users = realm.get_admin_users()
 
         if users:
@@ -32,4 +26,5 @@ class Command(BaseCommand):
         else:
             print('There are no admins for this realm!')
 
-        print('\nYou can use the "knight" management command to knight admins.')
+        print('\nYou can use the "knight" management command to make more users admins.')
+        print('\nOr with the --revoke argument, remove admin status from users.')

zerver/tests/test_auth_backends.py

@@ -874,9 +874,9 @@ class GoogleSubdomainLoginTest(GoogleOAuthTest):
 
     def test_google_oauth2_success(self) -> None:
         token_response = ResponseMock(200, {'access_token': "unique_token"})
-        account_data = dict(name=dict(formatted="Full Name"),
-                            emails=[dict(type="account",
-                                         value=self.example_email("hamlet"))])
+        account_data = dict(name="Full Name",
+                            email_verified=True,
+                            email=self.example_email("hamlet"))
         account_response = ResponseMock(200, account_data)
         result = self.google_oauth2_test(token_response, account_response,
                                          subdomain='zulip', next='/user_uploads/image')
@@ -892,24 +892,15 @@ class GoogleSubdomainLoginTest(GoogleOAuthTest):
                                  parsed_url.path)
         self.assertTrue(uri.startswith('http://zulip.testserver/accounts/login/subdomain/'))
 
-    def test_google_oauth2_no_fullname(self) -> None:
+    def test_user_cannot_log_without_verified_email(self) -> None:
         token_response = ResponseMock(200, {'access_token': "unique_token"})
-        account_data = dict(name=dict(givenName="Test", familyName="User"),
-                            emails=[dict(type="account",
-                                         value=self.example_email("hamlet"))])
+        account_data = dict(name="Full Name",
+                            email_verified=False,
+                            email=self.example_email("hamlet"))
         account_response = ResponseMock(200, account_data)
-        result = self.google_oauth2_test(token_response, account_response, subdomain='zulip')
-
-        data = load_subdomain_token(result)
-        self.assertEqual(data['email'], self.example_email("hamlet"))
-        self.assertEqual(data['name'], 'Test User')
-        self.assertEqual(data['subdomain'], 'zulip')
-        self.assertEqual(data['next'], '')
-        self.assertEqual(result.status_code, 302)
-        parsed_url = urllib.parse.urlparse(result.url)
-        uri = "{}://{}{}".format(parsed_url.scheme, parsed_url.netloc,
-                                 parsed_url.path)
-        self.assertTrue(uri.startswith('http://zulip.testserver/accounts/login/subdomain/'))
+        result = self.google_oauth2_test(token_response, account_response,
+                                         subdomain='zulip')
+        self.assertEqual(result.status_code, 400)
 
     def test_google_oauth2_mobile_success(self) -> None:
         self.user_profile = self.example_user('hamlet')
@@ -917,9 +908,9 @@ class GoogleSubdomainLoginTest(GoogleOAuthTest):
         self.user_profile.save()
         mobile_flow_otp = '1234abcd' * 8
         token_response = ResponseMock(200, {'access_token': "unique_token"})
-        account_data = dict(name=dict(formatted="Full Name"),
-                            emails=[dict(type="account",
-                                         value=self.user_profile.email)])
+        account_data = dict(name="Full Name",
+                            email_verified=True,
+                            email=self.user_profile.email)
         account_response = ResponseMock(200, account_data)
         self.assertEqual(len(mail.outbox), 0)
 
@@ -1137,9 +1128,9 @@ class GoogleSubdomainLoginTest(GoogleOAuthTest):
 
     def test_user_cannot_log_into_nonexisting_realm(self) -> None:
         token_response = ResponseMock(200, {'access_token': "unique_token"})
-        account_data = dict(name=dict(formatted="Full Name"),
-                            emails=[dict(type="account",
-                                         value=self.example_email("hamlet"))])
+        account_data = dict(name="Full Name",
+                            email_verified=True,
+                            email=self.example_email("hamlet"))
         account_response = ResponseMock(200, account_data)
         result = self.google_oauth2_test(token_response, account_response,
                                          subdomain='nonexistent')
@@ -1148,9 +1139,9 @@ class GoogleSubdomainLoginTest(GoogleOAuthTest):
 
     def test_user_cannot_log_into_wrong_subdomain(self) -> None:
         token_response = ResponseMock(200, {'access_token': "unique_token"})
-        account_data = dict(name=dict(formatted="Full Name"),
-                            emails=[dict(type="account",
-                                         value=self.example_email("hamlet"))])
+        account_data = dict(name="Full Name",
+                            email_verified=True,
+                            email=self.example_email("hamlet"))
         account_response = ResponseMock(200, account_data)
         result = self.google_oauth2_test(token_response, account_response,
                                          subdomain='zephyr')
@@ -1175,9 +1166,9 @@ class GoogleSubdomainLoginTest(GoogleOAuthTest):
         email = "newuser@zulip.com"
         realm = get_realm("zulip")
         token_response = ResponseMock(200, {'access_token': "unique_token"})
-        account_data = dict(name=dict(formatted="Full Name"),
-                            emails=[dict(type="account",
-                                         value=email)])
+        account_data = dict(name="Full Name",
+                            email_verified=True,
+                            email=email)
         account_response = ResponseMock(200, account_data)
         result = self.google_oauth2_test(token_response, account_response, subdomain='zulip',
                                          is_signup='1')
@@ -1267,17 +1258,6 @@ class GoogleLoginTest(GoogleOAuthTest):
         self.assertEqual(m.call_args_list[0][0][0],
                          "Google login failed making API call: Response text")
 
-    def test_google_oauth2_account_response_no_email(self) -> None:
-        token_response = ResponseMock(200, {'access_token': "unique_token"})
-        account_data = dict(name=dict(formatted="Full Name"),
-                            emails=[])
-        account_response = ResponseMock(200, account_data)
-        with mock.patch("logging.error") as m:
-            result = self.google_oauth2_test(token_response, account_response,
-                                             subdomain="zulip")
-        self.assertEqual(result.status_code, 400)
-        self.assertIn("Google oauth2 account email not found:", m.call_args_list[0][0][0])
-
     def test_google_oauth2_error_access_denied(self) -> None:
         result = self.client_get("/accounts/login/google/done/?error=access_denied")
         self.assertEqual(result.status_code, 302)

zerver/tests/test_push_notifications.py

@@ -668,10 +668,15 @@ class TestAPNs(PushNotificationTest):
                 mock.patch('zerver.lib.push_notifications.logging') as mock_logging:
             mock_get.return_value = None
             self.send()
-            mock_logging.warning.assert_called_once_with(
+            mock_logging.debug.assert_called_once_with(
                 "APNs: Dropping a notification because nothing configured.  "
                 "Set PUSH_NOTIFICATION_BOUNCER_URL (or APNS_CERT_FILE).")
-            mock_logging.info.assert_not_called()
+            mock_logging.warning.assert_not_called()
+            from zerver.lib.push_notifications import initialize_push_notifications
+            initialize_push_notifications()
+            mock_logging.warning.assert_called_once_with(
+                "Mobile push notifications are not configured.\n  "
+                "See https://zulip.readthedocs.io/en/latest/production/mobile-push-notifications.html")
 
     def test_success(self) -> None:
         with self.mock_apns() as mock_apns, \
@@ -1154,11 +1159,11 @@ class GCMTest(PushNotificationTest):
         return data
 
 class GCMNotSetTest(GCMTest):
-    @mock.patch('logging.warning')
-    def test_gcm_is_none(self, mock_warning: mock.MagicMock) -> None:
+    @mock.patch('logging.debug')
+    def test_gcm_is_none(self, mock_debug: mock.MagicMock) -> None:
         apn.gcm = None
         apn.send_android_push_notification_to_user(self.user_profile, {})
-        mock_warning.assert_called_with(
+        mock_debug.assert_called_with(
             "Skipping sending a GCM push notification since PUSH_NOTIFICATION_BOUNCER_URL "
             "and ANDROID_GCM_API_KEY are both unset")
 

zerver/tests/test_slack_importer.py

@@ -20,6 +20,8 @@ from zerver.data_import.slack import (
     do_convert_data,
     process_avatars,
     process_message_files,
+    AddedChannelsT,
+    ZerverFieldsT,
 )
 from zerver.data_import.import_util import (
     build_zerver_realm,
@@ -55,7 +57,7 @@ import shutil
 import requests
 import os
 import mock
-from typing import Any, AnyStr, Dict, List, Optional, Set, Tuple
+from typing import Any, AnyStr, Dict, List, Optional, Set, Tuple, Iterator
 
 def remove_folder(path: str) -> None:
     if os.path.exists(path):
@@ -405,7 +407,7 @@ class SlackImporter(ZulipTestCase):
         self.assertEqual(zerver_usermessage[3]['id'], um_id + 4)
         self.assertEqual(zerver_usermessage[3]['message'], message_id)
 
-    @mock.patch("zerver.data_import.slack.build_usermessages", return_value = 2)
+    @mock.patch("zerver.data_import.slack.build_usermessages", return_value = (2, 4))
     def test_channel_message_to_zerver_message(self, mock_build_usermessage: mock.Mock) -> None:
 
         user_data = [{"id": "U066MTL5U", "name": "john doe", "deleted": False, "real_name": "John"},
@@ -446,7 +448,7 @@ class SlackImporter(ZulipTestCase):
             channel_message_to_zerver_message(
                 1, user_data, added_users, added_recipient,
                 all_messages, [], subscriber_map,
-                added_channels, 'domain')
+                added_channels, 'domain', set())
         # functioning already tested in helper function
         self.assertEqual(zerver_usermessage, [])
         # subtype: channel_join is filtered
@@ -483,14 +485,19 @@ class SlackImporter(ZulipTestCase):
         self.assertEqual(zerver_message[3]['sender'], 24)
 
     @mock.patch("zerver.data_import.slack.channel_message_to_zerver_message")
-    @mock.patch("zerver.data_import.slack.get_all_messages")
-    def test_convert_slack_workspace_messages(self, mock_get_all_messages: mock.Mock,
+    @mock.patch("zerver.data_import.slack.get_messages_iterator")
+    def test_convert_slack_workspace_messages(self, mock_get_messages_iterator: mock.Mock,
                                               mock_message: mock.Mock) -> None:
         os.makedirs('var/test-slack-import', exist_ok=True)
         added_channels = {'random': ('c5', 1), 'general': ('c6', 2)}  # type: Dict[str, Tuple[str, int]]
+
         time = float(timezone_now().timestamp())
         zerver_message = [{'id': 1, 'ts': time}, {'id': 5, 'ts': time}]
 
+        def fake_get_messages_iter(slack_data_dir: str, added_channels: AddedChannelsT) -> Iterator[ZerverFieldsT]:
+            import copy
+            return iter(copy.deepcopy(zerver_message))
+
         realm = {'zerver_subscription': []}  # type: Dict[str, Any]
         user_list = []  # type: List[Dict[str, Any]]
         reactions = [{"name": "grinning", "users": ["U061A5N1G"], "count": 1}]
@@ -498,14 +505,15 @@ class SlackImporter(ZulipTestCase):
 
         zerver_usermessage = [{'id': 3}, {'id': 5}, {'id': 6}, {'id': 9}]
 
-        mock_get_all_messages.side_effect = [zerver_message]
+        mock_get_messages_iterator.side_effect = fake_get_messages_iter
         mock_message.side_effect = [[zerver_message[:1], zerver_usermessage[:2],
                                      attachments, uploads, reactions[:1]],
                                     [zerver_message[1:2], zerver_usermessage[2:5],
                                      attachments, uploads, reactions[1:1]]]
+        # Hacky: We should include a zerver_userprofile, not the empty []
         test_reactions, uploads, zerver_attachment = convert_slack_workspace_messages(
             './random_path', user_list, 2, {}, {}, added_channels,
-            realm, [], 'domain', 'var/test-slack-import', chunk_size=1)
+            realm, [], [], 'domain', 'var/test-slack-import', chunk_size=1)
         messages_file_1 = os.path.join('var', 'test-slack-import', 'messages-000001.json')
         self.assertTrue(os.path.exists(messages_file_1))
         messages_file_2 = os.path.join('var', 'test-slack-import', 'messages-000002.json')

zerver/views/auth.py

@@ -348,6 +348,7 @@ def send_oauth_request_to_google(request: HttpRequest) -> HttpResponse:
         'redirect_uri': reverse_on_root('zerver.views.auth.finish_google_oauth2'),
         'scope': 'profile email',
         'state': csrf_state,
+        'prompt': 'select_account',
     }
     return redirect(google_uri + urllib.parse.urlencode(params))
 
@@ -394,7 +395,7 @@ def finish_google_oauth2(request: HttpRequest) -> HttpResponse:
     access_token = resp.json()['access_token']
 
     resp = requests.get(
-        'https://www.googleapis.com/plus/v1/people/me',
+        'https://www.googleapis.com/oauth2/v3/userinfo',
         params={'access_token': access_token}
     )
     if resp.status_code == 400:
@@ -405,21 +406,13 @@ def finish_google_oauth2(request: HttpRequest) -> HttpResponse:
         return HttpResponse(status=400)
     body = resp.json()
 
-    try:
-        full_name = body['name']['formatted']
-    except KeyError:
-        # Only google+ users have a formatted name. I am ignoring i18n here.
-        full_name = '{} {}'.format(
-            body['name']['givenName'], body['name']['familyName']
-        )
-    for email in body['emails']:
-        if email['type'] == 'account':
-            break
-    else:
-        logging.error('Google oauth2 account email not found: %s' % (body,))
+    if not body['email_verified']:
+        logging.error('Google oauth2 account email not verified.')
         return HttpResponse(status=400)
 
-    email_address = email['value']
+    # Extract the user info from the Google response
+    full_name = body['name']
+    email_address = body['email']
 
     try:
         realm = Realm.objects.get(string_id=subdomain)

zerver/worker/queue_processors.py

@@ -23,7 +23,8 @@ from zerver.lib.feedback import handle_feedback
 from zerver.lib.queue import SimpleQueueClient, queue_json_publish, retry_event
 from zerver.lib.timestamp import timestamp_to_datetime
 from zerver.lib.notifications import handle_missedmessage_emails
-from zerver.lib.push_notifications import handle_push_notification, handle_remove_push_notification
+from zerver.lib.push_notifications import handle_push_notification, handle_remove_push_notification, \
+    initialize_push_notifications
 from zerver.lib.actions import do_send_confirmation_email, \
     do_update_user_activity, do_update_user_activity_interval, do_update_user_presence, \
     internal_send_message, check_send_message, extract_recipients, \
@@ -352,6 +353,13 @@ class MissedMessageSendingWorker(EmailSendingWorker):  # nocoverage
 
 @assign_queue('missedmessage_mobile_notifications')
 class PushNotificationsWorker(QueueProcessingWorker):  # nocoverage
+    def start(self) -> None:
+        # initialize_push_notifications doesn't strictly do anything
+        # beyond printing some logging warnings if push notifications
+        # are not available in the current configuration.
+        initialize_push_notifications()
+        super().start()
+
     def consume(self, data: Mapping[str, Any]) -> None:
         if data.get("type", "add") == "remove":
             handle_remove_push_notification(data['user_profile_id'], data['message_id'])

zproject/prod_settings_template.py

@@ -71,8 +71,13 @@ ZULIP_ADMINISTRATOR = 'zulip-admin@example.com'
 # The noreply address to be used as the sender for certain generated
 # emails.  Messages sent to this address could contain sensitive user
 # data and should not be delivered anywhere.  The default is
-# e.g. noreply@zulip.example.com (if EXTERNAL_HOST is
-# zulip.example.com).
+# e.g. noreply-{random_token}@zulip.example.com (if EXTERNAL_HOST is
+# zulip.example.com).  There are potential security issues if you set
+# ADD_TOKENS_TO_NOREPLY_ADDRESS=False to remove the token; see
+# https://zulip.readthedocs.io/en/latest/production/email.html for details.
+#ADD_TOKENS_TO_NOREPLY_ADDRESS = True
+#TOKENIZED_NOREPLY_EMAIL_ADDRESS = "noreply-{token}@example.com"
+# Used for noreply emails only if ADD_TOKENS_TO_NOREPLY_ADDRESS=False
 #NOREPLY_EMAIL_ADDRESS = 'noreply@example.com'
 
 # Many countries and bulk mailers require certain types of email to display