Release Zulip Server 3.2

(cherry picked from commit 7b9fe77bf1)

docs/overview/changelog.md

@@ -7,6 +7,28 @@ All notable changes to the Zulip server are documented in this file.
 This section lists notable unreleased changes; it is generally updated
 in bursts.
 
+### 3.2 -- September 15, 2020
+
+- Switched from `libmemcached` to `python-binary-memcached`, a
+  pure-Python implementation; this should eliminate memcached
+  connection problems affecting some installations.
+- Removed unnecessary `django-cookies-samesite` dependency, which had
+  its latest release removed from PyPI (breaking installation of Zulip
+  3.1).
+- Limited which local email addresses Postfix accepts when the
+  incoming email integration is enabled; this prevents the enumeration
+  of local users via the email system.
+- Fixed incorrectly case-sensitive email validation in `REMOTE_USER`
+  authentication.
+- Fixed search results for `has:image`.
+- Fixed ability to adjust "Who can post on the stream" configuration.
+- Fixed display of "Permission [to post] will be granted in n days"
+  for n > 365.
+- Support providing `nginx_listen_port` setting in conjunction with
+  `http_only` in `zulip.conf`.
+- Improved upgrade documentation.
+- Removed internal ID lists which could leak into the events API.
+
 ### 3.1 -- July 30, 2020
 
 - Removed unused `short_name` field from the User model.  This field

docs/production/export-and-import.md

@@ -65,8 +65,9 @@ su zulip -c '/home/zulip/deployments/current/manage.py backup'
 ```
 
 The backup tool provides the following options:
-- `--output`: Path where the output file should be stored. If no path is
- provided, the output file is saved to a temporary directory.
+- `--output=/tmp/backup.tar.gz`: Filename to write the backup tarball
+  to (default: write to a file in `/tmp`).  On success, the
+  console output will show the path to the output tarball.
 - `--skip-db`: Skip backup of the database.  Useful if you're using a
   remote postgres host with its own backup system and just need to
   backup non-database state.

docs/production/upgrade-or-modify.md

@@ -249,9 +249,9 @@ instructions for other supported platforms.
         /home/zulip/deployments/current/ --ignore-static-assets --audit-fts-indexes
     ```
 
-That last command will finish by restarting your Zulip server; you
-should now be able to navigate to its URL and confirm everything is
-working correctly.
+   This will finish by restarting your Zulip server; you should now be
+   able to navigate to its URL and confirm everything is working
+   correctly.
 
 ### Upgrading from Ubuntu 16.04 Xenial to 18.04 Bionic
 
@@ -278,11 +278,28 @@ working correctly.
     systemctl restart memcached
     ```
 
-5. Same as for Bionic to Focal.
+5. Finally, we need to reinstall the current version of Zulip, which
+   among other things will recompile Zulip's Python module
+   dependencies for your new version of Python:
 
-That last command will finish by restarting your Zulip server; you
-should now be able to navigate to its URL and confirm everything is
-working correctly.
+    ```
+    rm -rf /srv/zulip-venv-cache/*
+    /home/zulip/deployments/current/scripts/lib/upgrade-zulip-stage-2 \
+        /home/zulip/deployments/current/ --ignore-static-assets
+    ```
+
+   This will finish by restarting your Zulip server; you should now
+   be able to navigate to its URL and confirm everything is working
+   correctly.
+
+6. [Upgrade to the latest Zulip release](#upgrading-to-a-release), now
+   that your server is running a supported operating system.
+
+7. As root, finish by verifying the contents of the full-text indexes:
+
+    ```
+    /home/zulip/deployments/current/manage.py audit_fts_indexes
+    ```
 
 ### Upgrading from Ubuntu 14.04 Trusty to 16.04 Xenial
 
@@ -295,7 +312,7 @@ working correctly.
 3. Same as for Bionic to Focal.
 
 4. As root, upgrade the database installation and OS configuration to
-match the new OS version:
+   match the new OS version:
 
     ```
     apt remove upstart -y
@@ -309,11 +326,23 @@ match the new OS version:
     service memcached restart
     ```
 
-5. Same as for Bionic to Focal.
+5. Finally, we need to reinstall the current version of Zulip, which
+   among other things will recompile Zulip's Python module
+   dependencies for your new version of Python:
 
-That last command will finish by restarting your Zulip server; you
-should now be able to navigate to its URL and confirm everything is
-working correctly.
+    ```
+    rm -rf /srv/zulip-venv-cache/*
+    /home/zulip/deployments/current/scripts/lib/upgrade-zulip-stage-2 \
+        /home/zulip/deployments/current/ --ignore-static-assets
+    ```
+
+   This will finish by restarting your Zulip server; you should now be
+   able to navigate to its URL and confirm everything is working
+   correctly.
+
+6. [Upgrade from Xenial to
+   Bionic](#upgrading-from-ubuntu-16-04-xenial-to-18-04-bionic), so
+   that you are running a supported operating system.
 
 ### Upgrading from Debian Stretch to Debian Buster
 
@@ -348,11 +377,28 @@ working correctly.
     service memcached restart
     ```
 
-5. Same as for Bionic to Focal.
+5. Finally, we need to reinstall the current version of Zulip, which
+   among other things will recompile Zulip's Python module
+   dependencies for your new version of Python:
 
-That last command will finish by restarting your Zulip server; you
-should now be able to navigate to its URL and confirm everything is
-working correctly.
+    ```
+    rm -rf /srv/zulip-venv-cache/*
+    /home/zulip/deployments/current/scripts/lib/upgrade-zulip-stage-2 \
+        /home/zulip/deployments/current/ --ignore-static-assets
+    ```
+
+   This will finish by restarting your Zulip server; you should now
+   be able to navigate to its URL and confirm everything is working
+   correctly.
+
+6. [Upgrade to the latest Zulip release](#upgrading-to-a-release), now
+   that your server is running a supported operating system.
+
+7. As root, finish by verifying the contents of the full-text indexes:
+
+    ```
+    /home/zulip/deployments/current/manage.py audit_fts_indexes
+    ```
 
 ## Upgrading PostgreSQL
 

frontend_tests/node_tests/filter.js

@@ -720,34 +720,40 @@ run_test("predicate_basics", () => {
     // HTML content of message is used to determine if image have link, image or attachment.
     // We are using jquery to parse the html and find existence of relevant tags/elements.
     // In tests we need to stub the calls to jquery so using zjquery's .set_find_results method.
+    function set_find_results_for_msg_content(msg, jquery_selector, results) {
+        $(`<div>${msg.content}</div>`).set_find_results(jquery_selector, results);
+    }
+
     const has_link = get_predicate([["has", "link"]]);
-    $(img_msg.content).set_find_results("a", [$("<a>")]);
+    set_find_results_for_msg_content(img_msg, "a", [$("<a>")]);
     assert(has_link(img_msg));
-    $(non_img_attachment_msg.content).set_find_results("a", [$("<a>")]);
+    set_find_results_for_msg_content(non_img_attachment_msg, "a", [$("<a>")]);
     assert(has_link(non_img_attachment_msg));
-    $(link_msg.content).set_find_results("a", [$("<a>")]);
+    set_find_results_for_msg_content(link_msg, "a", [$("<a>")]);
     assert(has_link(link_msg));
-    $(no_has_filter_matching_msg.content).set_find_results("a", false);
+    set_find_results_for_msg_content(no_has_filter_matching_msg, "a", false);
     assert(!has_link(no_has_filter_matching_msg));
 
     const has_attachment = get_predicate([["has", "attachment"]]);
-    $(img_msg.content).set_find_results("a[href^='/user_uploads']", [$("<a>")]);
+    set_find_results_for_msg_content(img_msg, "a[href^='/user_uploads']", [$("<a>")]);
     assert(has_attachment(img_msg));
-    $(non_img_attachment_msg.content).set_find_results("a[href^='/user_uploads']", [$("<a>")]);
+    set_find_results_for_msg_content(non_img_attachment_msg, "a[href^='/user_uploads']", [
+        $("<a>"),
+    ]);
     assert(has_attachment(non_img_attachment_msg));
-    $(link_msg.content).set_find_results("a[href^='/user_uploads']", false);
+    set_find_results_for_msg_content(link_msg, "a[href^='/user_uploads']", false);
     assert(!has_attachment(link_msg));
-    $(no_has_filter_matching_msg.content).set_find_results("a[href^='/user_uploads']", false);
+    set_find_results_for_msg_content(no_has_filter_matching_msg, "a[href^='/user_uploads']", false);
     assert(!has_attachment(no_has_filter_matching_msg));
 
     const has_image = get_predicate([["has", "image"]]);
-    $(img_msg.content).set_find_results(".message_inline_image", [$("<img>")]);
+    set_find_results_for_msg_content(img_msg, ".message_inline_image", [$("<img>")]);
     assert(has_image(img_msg));
-    $(non_img_attachment_msg.content).set_find_results(".message_inline_image", false);
+    set_find_results_for_msg_content(non_img_attachment_msg, ".message_inline_image", false);
     assert(!has_image(non_img_attachment_msg));
-    $(link_msg.content).set_find_results(".message_inline_image", false);
+    set_find_results_for_msg_content(link_msg, ".message_inline_image", false);
     assert(!has_image(link_msg));
-    $(no_has_filter_matching_msg.content).set_find_results(".message_inline_image", false);
+    set_find_results_for_msg_content(no_has_filter_matching_msg, ".message_inline_image", false);
     assert(!has_image(no_has_filter_matching_msg));
 });
 

puppet/zulip/files/postfix/access

@@ -0,0 +1,9 @@
+# This is the list of email addresses that are accepted via SMTP;
+# these consist of only the addresses in `virtual`, as well as the
+# RFC822-specified postmaster.
+
+/\+.*@/         OK
+/\..*@/         OK
+/^mm/           OK
+
+/^postmaster@/  OK

puppet/zulip/files/postfix/virtual

@@ -1,3 +1,6 @@
-/\+.*@/ zulip@localhost
-/\..*@/ zulip@localhost
-/^mm/ zulip@localhost
+# Changes to this list require a corresponding change to `access` as
+# well.
+
+/\+.*@/  zulip@localhost
+/\..*@/  zulip@localhost
+/^mm/    zulip@localhost

puppet/zulip/manifests/app_frontend.pp

@@ -5,7 +5,11 @@ class zulip::app_frontend {
   include zulip::app_frontend_once
 
   $nginx_http_only = zulipconf('application_server', 'http_only', undef)
-  $nginx_listen_port = zulipconf('application_server', 'nginx_listen_port', 443)
+  if $nginx_http_only != '' {
+    $nginx_listen_port = zulipconf('application_server', 'nginx_listen_port', 80)
+  } else {
+    $nginx_listen_port = zulipconf('application_server', 'nginx_listen_port', 443)
+  }
   $no_serve_uploads = zulipconf('application_server', 'no_serve_uploads', undef)
   $ssl_dir = $::osfamily ? {
     'debian' => '/etc/ssl',

puppet/zulip/manifests/postfix_localmail.pp

@@ -67,4 +67,12 @@ class zulip::postfix_localmail {
     ],
   }
 
+  file {'/etc/postfix/access':
+    ensure  => file,
+    mode    => '0644',
+    owner   => root,
+    group   => root,
+    source  => 'puppet:///modules/zulip/postfix/access',
+    require => Package[postfix],
+  }
 }

puppet/zulip/templates/nginx/zulip-enterprise.template.erb

@@ -16,8 +16,8 @@ include /etc/nginx/zulip-include/upstreams;
 
 server {
 <% if @nginx_http_only != '' -%>
-    listen 80;
-    listen [::]:80;
+    listen <%= @nginx_listen_port %>;
+    listen [::]:<%= @nginx_listen_port %>;
 <% else -%>
     listen <%= @nginx_listen_port %> http2;
     listen [::]:<%= @nginx_listen_port %> http2;

puppet/zulip/templates/postfix/main.cf.erb

@@ -16,6 +16,7 @@ smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
 smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
 
 smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated reject_unauth_destination
+smtpd_recipient_restrictions = check_recipient_access regexp:/etc/postfix/access, reject
 myhostname = <%= @fqdn %>
 alias_maps = hash:/etc/aliases
 alias_database = hash:/etc/aliases

requirements/common.in

@@ -178,9 +178,6 @@ pyahocorasick
 # Used for rate limiting authentication.
 decorator
 
-# Use SameSite cookies in legacy Django (remove with Django 2.1)
-django-cookies-samesite
-
 # For server-side enforcement of password strength
 zxcvbn
 

requirements/dev.txt

@@ -276,9 +276,6 @@ django-bitfield==2.0.1 \
 django-bmemcached==0.3.0 \
     --hash=sha256:4e4b7d97216dbae331c1de10e699ca22804b94ec3a90d2762dd5d146e6986a8a \
     # via -r requirements/common.in
-django-cookies-samesite==0.6.6 \
-    --hash=sha256:a26dc27bfc446279c981a301b053eff845b93d9ba62798e281c90584a7ccaa4a \
-    # via -r requirements/common.in
 django-formtools==2.2 \
     --hash=sha256:304fa777b8ef9e0693ce7833f885cb89ba46b0e46fc23b01176900a93f46742f \
     --hash=sha256:c5272c03c1cd51b2375abf7397a199a3148a9fbbf2f100e186467a84025d13b2 \
@@ -920,8 +917,6 @@ sh==1.12.14 \
 six==1.15.0 \
     --hash=sha256:30639c035cdb23534cd4aa2dd52c3bf48f06e5f4a941509c8bafd8ce11080259 \
     --hash=sha256:8b74bedcbbbaca38ff6d7491d76f2b06b3592611af620f8426e82dddb04a5ced \
-    --hash=sha256:8ce375b18ae4a749516d7e6c6fbbf8be6177c53974f53534d8eadb646cd279b1 \
-    --hash=sha256:92ad876fb6a201a7e23a6b85ea96d9643a51e285667c253a8653643804f7cb68 \
     # via argon2-cffi, automat, aws-sam-translator, cfn-lint, cryptography, django-bitfield, docker, ecdsa, hypchat, isodate, jsonschema, junit-xml, libthumbor, moto, openapi-core, openapi-schema-validator, openapi-spec-validator, packaging, parsel, pip-tools, protego, pyopenssl, python-binary-memcached, python-dateutil, python-debian, python-jose, qrcode, responses, social-auth-app-django, social-auth-core, talon, traitlets, twilio, w3lib, websocket-client, zulip
 snakeviz==2.1.0 \
     --hash=sha256:8ce375b18ae4a749516d7e6c6fbbf8be6177c53974f53534d8eadb646cd279b1 \
@@ -1108,10 +1103,6 @@ typing-extensions==3.7.4.2 \
     --hash=sha256:79ee589a3caca649a9bfd2a8de4709837400dfa00b6cc81962a1e6a1815969ae \
     --hash=sha256:f8d2bd89d25bc39dabe7d23df520442fa1d8969b82544370e03d88b5a591c392 \
     # via -r requirements/common.in, mypy, zulint
-ua-parser==0.10.0 \
-    --hash=sha256:46ab2e383c01dbd2ab284991b87d624a26a08f72da4d7d413f5bfab8b9036f8a \
-    --hash=sha256:47b1782ed130d890018d983fac37c2a80799d9e0b9c532e734c67cf70f185033 \
-    # via django-cookies-samesite
 uhashring==1.2 \
     --hash=sha256:f7304ca2ff763bbf1e2f8a78f21131721811619c5841de4f8c98063344906931 \
     # via python-binary-memcached

requirements/docs.txt

@@ -106,8 +106,6 @@ requests==2.24.0 \
 six==1.15.0 \
     --hash=sha256:30639c035cdb23534cd4aa2dd52c3bf48f06e5f4a941509c8bafd8ce11080259 \
     --hash=sha256:8b74bedcbbbaca38ff6d7491d76f2b06b3592611af620f8426e82dddb04a5ced \
-    --hash=sha256:8ce375b18ae4a749516d7e6c6fbbf8be6177c53974f53534d8eadb646cd279b1 \
-    --hash=sha256:92ad876fb6a201a7e23a6b85ea96d9643a51e285667c253a8653643804f7cb68 \
     # via packaging
 snowballstemmer==2.0.0 \
     --hash=sha256:209f257d7533fdb3cb73bdbd24f436239ca3b2fa67d56f6ff88e86be08cc5ef0 \

requirements/prod.txt

@@ -188,9 +188,6 @@ django-bitfield==2.0.1 \
 django-bmemcached==0.3.0 \
     --hash=sha256:4e4b7d97216dbae331c1de10e699ca22804b94ec3a90d2762dd5d146e6986a8a \
     # via -r requirements/common.in
-django-cookies-samesite==0.6.6 \
-    --hash=sha256:a26dc27bfc446279c981a301b053eff845b93d9ba62798e281c90584a7ccaa4a \
-    # via -r requirements/common.in
 django-formtools==2.2 \
     --hash=sha256:304fa777b8ef9e0693ce7833f885cb89ba46b0e46fc23b01176900a93f46742f \
     --hash=sha256:c5272c03c1cd51b2375abf7397a199a3148a9fbbf2f100e186467a84025d13b2 \
@@ -656,8 +653,6 @@ s3transfer==0.3.3 \
 six==1.15.0 \
     --hash=sha256:30639c035cdb23534cd4aa2dd52c3bf48f06e5f4a941509c8bafd8ce11080259 \
     --hash=sha256:8b74bedcbbbaca38ff6d7491d76f2b06b3592611af620f8426e82dddb04a5ced \
-    --hash=sha256:8ce375b18ae4a749516d7e6c6fbbf8be6177c53974f53534d8eadb646cd279b1 \
-    --hash=sha256:92ad876fb6a201a7e23a6b85ea96d9643a51e285667c253a8653643804f7cb68 \
     # via argon2-cffi, cryptography, django-bitfield, hypchat, isodate, jsonschema, libthumbor, openapi-core, openapi-schema-validator, openapi-spec-validator, pyopenssl, python-binary-memcached, python-dateutil, qrcode, social-auth-app-django, social-auth-core, talon, traitlets, twilio, zulip
 social-auth-app-django==4.0.0 \
     --hash=sha256:2c69e57df0b30c9c1823519c5f1992cbe4f3f98fdc7d95c840e091a752708840 \
@@ -744,10 +739,6 @@ typing-extensions==3.7.4.2 \
     --hash=sha256:79ee589a3caca649a9bfd2a8de4709837400dfa00b6cc81962a1e6a1815969ae \
     --hash=sha256:f8d2bd89d25bc39dabe7d23df520442fa1d8969b82544370e03d88b5a591c392 \
     # via -r requirements/common.in
-ua-parser==0.10.0 \
-    --hash=sha256:46ab2e383c01dbd2ab284991b87d624a26a08f72da4d7d413f5bfab8b9036f8a \
-    --hash=sha256:47b1782ed130d890018d983fac37c2a80799d9e0b9c532e734c67cf70f185033 \
-    # via django-cookies-samesite
 uhashring==1.2 \
     --hash=sha256:f7304ca2ff763bbf1e2f8a78f21131721811619c5841de4f8c98063344906931 \
     # via python-binary-memcached

scripts/lib/setup_venv.py

@@ -34,6 +34,8 @@ VENV_DEPENDENCIES = [
     # on upgrade of a production server, and it's not worth adding
     # another call to `apt install` for.
     "jq",                   # Used by scripts/lib/install-node to check yarn version
+
+    "libsasl2-dev",         # For building python-ldap from source
 ]
 
 COMMON_YUM_VENV_DEPENDENCIES = [

static/js/compose.js

@@ -538,7 +538,7 @@ function validate_stream_message_post_policy(sub) {
     const person = people.get_by_user_id(page_params.user_id);
     const current_datetime = new Date(Date.now());
     const person_date_joined = new Date(person.date_joined);
-    const days = new Date(current_datetime - person_date_joined).getDate();
+    const days = (current_datetime - person_date_joined) / 1000 / 86400;
     let error_text;
     if (
         stream_post_policy === stream_post_permission_type.non_new_members.code &&

static/js/message_util.js

@@ -16,16 +16,25 @@ function add_messages(messages, msg_list, opts) {
     return render_info;
 }
 
+// We need to check if the message content contains the specified HTML
+// elements.  We wrap the message.content in a <div>; this is
+// important because $("Text <a>link</a>").find("a") returns nothing;
+// one needs an outer element wrapping an object to use this
+// construction.
+function is_element_in_message_content(message, element_selector) {
+    return $(`<div>${message.content}</div>`).find(element_selector).length > 0;
+}
+
 exports.message_has_link = function (message) {
-    return $(message.content).find("a").length > 0;
+    return is_element_in_message_content(message, "a");
 };
 
 exports.message_has_image = function (message) {
-    return $(message.content).find(".message_inline_image").length > 0;
+    return is_element_in_message_content(message, ".message_inline_image");
 };
 
 exports.message_has_attachment = function (message) {
-    return $(message.content).find("a[href^='/user_uploads']").length > 0;
+    return is_element_in_message_content(message, "a[href^='/user_uploads']");
 };
 
 exports.add_old_messages = function (messages, msg_list) {

static/js/stream_edit.js

@@ -480,11 +480,22 @@ exports.set_stream_property = function (sub, property, value, status_element) {
     exports.bulk_set_stream_property([sub_data], status_element);
 };
 
+function get_message_retention_days_from_sub(sub) {
+    if (sub.message_retention_days === null) {
+        return "realm_default";
+    }
+    if (sub.message_retention_days === -1) {
+        return "forever";
+    }
+    return sub.message_retention_days;
+}
+
 function change_stream_privacy(e) {
     e.stopPropagation();
 
     const stream_id = $(e.target).data("stream-id");
     const sub = stream_data.get_sub_by_id(stream_id);
+    const data = {};
 
     const privacy_setting = $("#stream_privacy_modal input[name=privacy]:checked").val();
     const stream_post_policy = parseInt(
@@ -492,6 +503,10 @@ function change_stream_privacy(e) {
         10,
     );
 
+    if (sub.stream_post_policy !== stream_post_policy) {
+        data.stream_post_policy = JSON.stringify(stream_post_policy);
+    }
+
     let invite_only;
     let history_public_to_subscribers;
 
@@ -506,28 +521,38 @@ function change_stream_privacy(e) {
         history_public_to_subscribers = true;
     }
 
-    $(".stream_change_property_info").hide();
-    const data = {
-        stream_name: sub.name,
-        // toggle the privacy setting
-        is_private: JSON.stringify(invite_only),
-        stream_post_policy: JSON.stringify(stream_post_policy),
-        history_public_to_subscribers: JSON.stringify(history_public_to_subscribers),
-    };
+    if (
+        sub.invite_only !== invite_only ||
+        sub.history_public_to_subscribers !== history_public_to_subscribers
+    ) {
+        data.is_private = JSON.stringify(invite_only);
+        data.history_public_to_subscribers = JSON.stringify(history_public_to_subscribers);
+    }
 
-    if (page_params.is_owner) {
-        let message_retention_days = $(
-            "#stream_privacy_modal select[name=stream_message_retention_setting]",
-        ).val();
-        if (message_retention_days === "retain_for_period") {
-            message_retention_days = parseInt(
-                $("#stream_privacy_modal input[name=stream-message-retention-days]").val(),
-                10,
-            );
-        }
+    let message_retention_days = $(
+        "#stream_privacy_modal select[name=stream_message_retention_setting]",
+    ).val();
+    if (message_retention_days === "retain_for_period") {
+        message_retention_days = parseInt(
+            $("#stream_privacy_modal input[name=stream-message-retention-days]").val(),
+            10,
+        );
+    }
+
+    const message_retention_days_from_sub = get_message_retention_days_from_sub(sub);
+
+    if (message_retention_days_from_sub !== message_retention_days) {
         data.message_retention_days = JSON.stringify(message_retention_days);
     }
 
+    $(".stream_change_property_info").hide();
+
+    if (Object.keys(data).length === 0) {
+        overlays.close_modal("#stream_privacy_modal");
+        $("#stream_privacy_modal").remove();
+        return;
+    }
+
     channel.patch({
         url: "/json/streams/" + stream_id,
         data: data,

version.py

@@ -1,6 +1,6 @@
 import os
 
-ZULIP_VERSION = "3.1"
+ZULIP_VERSION = "3.2"
 # Add information on number of commits and commit hash to version, if available
 zulip_git_version_file = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'zulip-git-version')
 if os.path.exists(zulip_git_version_file):
@@ -10,7 +10,7 @@ if os.path.exists(zulip_git_version_file):
             ZULIP_VERSION = version
 
 LATEST_MAJOR_VERSION = "3.0"
-LATEST_RELEASE_VERSION = "3.1"
+LATEST_RELEASE_VERSION = "3.2"
 LATEST_RELEASE_ANNOUNCEMENT = "https://blog.zulip.org/2020/07/16/zulip-3-0-released/"
 LATEST_DESKTOP_VERSION = "5.3.0"
 
@@ -44,4 +44,4 @@ API_FEATURE_LEVEL = 27
 #   historical commits sharing the same major version, in which case a
 #   minor version bump suffices.
 
-PROVISION_VERSION = '91.0'
+PROVISION_VERSION = '92.0'

zerver/tests/test_auth_backends.py

@@ -3581,6 +3581,14 @@ class TestZulipRemoteUserBackend(DesktopFlowTestingLib, ZulipTestCase):
             self.assertEqual(result.status_code, 302)
             self.assert_logged_in_user_id(user_profile.id)
 
+    def test_login_case_insensitive(self) -> None:
+        user_profile = self.example_user('hamlet')
+        email_upper = user_profile.delivery_email.upper()
+        with self.settings(AUTHENTICATION_BACKENDS=('zproject.backends.ZulipRemoteUserBackend',)):
+            result = self.client_get('/accounts/login/sso/', REMOTE_USER=email_upper)
+            self.assertEqual(result.status_code, 302)
+            self.assert_logged_in_user_id(user_profile.id)
+
     def test_login_failure(self) -> None:
         email = self.example_email("hamlet")
         result = self.client_get('/accounts/login/sso/', REMOTE_USER=email)

zerver/tests/test_events.py

@@ -408,13 +408,6 @@ class NormalActionsTest(BaseAction):
             ('edit_timestamp', check_int),
             ('message_id', check_int),
             ('message_ids', check_list(check_int)),
-            ('prior_mention_user_ids', check_list(check_int)),
-            ('mention_user_ids', check_list(check_int)),
-            ('wildcard_mention_user_ids', check_list(check_int)),
-            ('presence_idle_user_ids', check_list(check_int)),
-            ('stream_push_user_ids', check_list(check_int)),
-            ('stream_email_user_ids', check_list(check_int)),
-            ('push_notify_user_ids', check_list(check_int)),
             ('orig_content', check_string),
             ('orig_rendered_content', check_string),
             (ORIG_TOPIC, check_string),

zerver/tests/test_signup.py

@@ -4098,15 +4098,16 @@ class TestFindMyTeam(ZulipTestCase):
         self.assertIn("Find your Zulip accounts", result.content.decode('utf8'))
 
     def test_result(self) -> None:
+        # We capitalize a letter in cordelia's email to test that the search is case-insensitive.
         result = self.client_post('/accounts/find/',
-                                  dict(emails="iago@zulip.com,cordelia@zulip.com"))
+                                  dict(emails="iago@zulip.com,cordeliA@zulip.com"))
         self.assertEqual(result.status_code, 302)
-        self.assertEqual(result.url, "/accounts/find/?emails=iago%40zulip.com%2Ccordelia%40zulip.com")
+        self.assertEqual(result.url, "/accounts/find/?emails=iago%40zulip.com%2CcordeliA%40zulip.com")
         result = self.client_get(result.url)
         content = result.content.decode('utf8')
         self.assertIn("Emails sent! You will only receive emails", content)
-        self.assertIn(self.example_email("iago"), content)
-        self.assertIn(self.example_email("cordelia"), content)
+        self.assertIn("iago@zulip.com", content)
+        self.assertIn("cordeliA@zulip.com", content)
         from django.core.mail import outbox
 
         # 3 = 1 + 2 -- Cordelia gets an email each for the "zulip" and "lear" realms.

zerver/tornado/event_queue.py

@@ -1007,15 +1007,18 @@ def process_deletion_event(event: Mapping[str, Any], users: Iterable[int]) -> No
                 del compatibility_event['message_ids']
                 client.add_event(compatibility_event)
 
-def process_message_update_event(event_template: Mapping[str, Any],
+def process_message_update_event(orig_event: Mapping[str, Any],
                                  users: Iterable[Mapping[str, Any]]) -> None:
-    prior_mention_user_ids = set(event_template.get('prior_mention_user_ids', []))
-    mention_user_ids = set(event_template.get('mention_user_ids', []))
-    presence_idle_user_ids = set(event_template.get('presence_idle_user_ids', []))
-    stream_push_user_ids = set(event_template.get('stream_push_user_ids', []))
-    stream_email_user_ids = set(event_template.get('stream_email_user_ids', []))
-    wildcard_mention_user_ids = set(event_template.get('wildcard_mention_user_ids', []))
-    push_notify_user_ids = set(event_template.get('push_notify_user_ids', []))
+    # Extract the parameters passed via the event object that don't
+    # belong in the actual events.
+    event_template = dict(orig_event)
+    prior_mention_user_ids = set(event_template.pop('prior_mention_user_ids', []))
+    mention_user_ids = set(event_template.pop('mention_user_ids', []))
+    presence_idle_user_ids = set(event_template.pop('presence_idle_user_ids', []))
+    stream_push_user_ids = set(event_template.pop('stream_push_user_ids', []))
+    stream_email_user_ids = set(event_template.pop('stream_email_user_ids', []))
+    wildcard_mention_user_ids = set(event_template.pop('wildcard_mention_user_ids', []))
+    push_notify_user_ids = set(event_template.pop('push_notify_user_ids', []))
 
     stream_name = event_template.get('stream_name')
     message_id = event_template['message_id']

zerver/views/registration.py

@@ -7,6 +7,7 @@ from django.conf import settings
 from django.contrib.auth import authenticate, get_backends
 from django.core import validators
 from django.core.exceptions import ValidationError
+from django.db.models import Q
 from django.http import HttpRequest, HttpResponse, HttpResponseRedirect
 from django.shortcuts import redirect, render
 from django.urls import reverse
@@ -601,8 +602,15 @@ def find_account(request: HttpRequest) -> HttpResponse:
         form = FindMyTeamForm(request.POST)
         if form.is_valid():
             emails = form.cleaned_data['emails']
+
+            # Django doesn't support __iexact__in lookup with EmailField, so we have
+            # to use Qs to get around that without needing to do multiple queries.
+            emails_q = Q()
+            for email in emails:
+                emails_q |= Q(delivery_email__iexact=email)
+
             for user in UserProfile.objects.filter(
-                    delivery_email__in=emails, is_active=True, is_bot=False,
+                    emails_q, is_active=True, is_bot=False,
                     realm__deactivated=False):
                 context = common_context(user)
                 context.update({

zproject/backends.py

@@ -988,7 +988,7 @@ class ExternalAuthResult:
         if self.user_profile is not None:
             # Ensure data inconsistent with the user_profile wasn't passed in inside the data_dict argument.
             assert 'full_name' not in data_dict or data_dict['full_name'] == self.user_profile.full_name
-            assert 'email' not in data_dict or data_dict['email'] == self.user_profile.delivery_email
+            assert 'email' not in data_dict or data_dict['email'].lower() == self.user_profile.delivery_email.lower()
             # Update these data_dict fields to ensure consistency with self.user_profile. This is mostly
             # defensive code, but is useful in these scenarios:
             # 1. user_profile argument was passed in, and no full_name or email_data in the data_dict arg.

zproject/computed_settings.py

@@ -393,8 +393,6 @@ REDIS_PASSWORD = get_secret('redis_password')
 # SECURITY SETTINGS
 ########################################################################
 
-SESSION_COOKIE_SAMESITE = 'Lax'
-
 # Tell the browser to never send our cookies without encryption, e.g.
 # when executing the initial http -> https redirect.
 #