Commit Graph

33634 Commits

Author SHA1 Message Date
Tim Abbott
4223254548 version: Update version and changelog for Zulip 2.0.6 release. 2019-09-23 16:22:41 -07:00
Anders Kaseorg
311aa21d9c styles: Add overflow: hidden on .message_content.
This has two purposes:

1. Prevent stupid stacks of diacritical marks from overflowing into
other messages.  Fixes #7843.

2. Prevent Chrome from collapsing the inside bottom margin with the
.messagebox outside (in a way that Firefox doesn’t).

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-23 16:09:58 -07:00
Anders Kaseorg
4fdc80a9c7 setup-apt-repo: Install groonga-keyring.
This allows the system to get updates to the Groonga repository
signing key, so `apt update` doesn’t start failing when the key
changes (like it recently did).

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-23 16:01:39 -07:00
Anders Kaseorg
d1e504079d setup-apt-repo: Don’t waste time installing debian-archive-keyring.
debian-archive-keyring is a dependency of the essential package apt,
so it is present in every Debian system.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-23 16:01:39 -07:00
Anders Kaseorg
2ff87bd888 setup: Update groonga APT repository signing key.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-23 16:01:39 -07:00
Joao Mauricio Carvalho
86d507db05 settings: Change username to bot email in bot settings HTML.
This is for consistency with how this value appears in other places in
the UI.

Fixes: #13162.
2019-09-23 15:55:25 -07:00
Tim Abbott
8024b1179a bots: Fix bot email addresses with EMAIL_ADDRESS_VISIBILITY_ADMINS.
When using our EMAIL_ADDRESS_VISIBILITY_ADMINS feature, we were
apparently creating bot users with different email and delivery_email
properties, due to effectively an oversight in how the code was
written (the initial migration handled bots correctly, but not bots
created after the transition).

Following the refactor in the last commit, the fix for this is just
adding the missing conditional, a test, and a database migration to
fix any incorrectly created bots leaked previously.
2019-09-23 15:53:53 -07:00
Tim Abbott
6e5c99328a models: Extract a helper function for whether emails are realm-public.
This is also a useful preparatory refactor for having a user setting
controlling whether one's own email address is publicly available
within the organization.
2019-09-23 15:53:53 -07:00
Anders Kaseorg
6bd977f9e4 requirements: Link to our GitHub issue for upgrading pika.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-23 14:30:18 -07:00
Anders Kaseorg
ba8a172897 requirements: Downgrade transifex-client so we can upgrade six, urllib3.
transifex-client 0.13.5 added overly strict version bounds on six and
urllib3.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-23 13:47:55 -07:00
Anders Kaseorg
f671ca3780 requirements: Upgrade Python requirements.
This commit was generated by deleting these lock files and rerunning
update-locked-requirements.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-23 13:23:58 -07:00
Anders Kaseorg
76492b25ae setup_venv: Install pip.txt requirements with --force-reinstall.
virtualenv on Ubuntu 16.04, when creating a new environment, downloads
the current version of setuptools, then replaces its pkg_resources
with an old copy from
/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl.
This causes problems, a simple example of which is reproducible from
the ubuntu:16.04 Docker base image as follows:

    apt-get update
    apt-get -y install python3-virtualenv
    python3 -m virtualenv -p python3 /ve
    /ve/bin/pip install sockjs-tornado
    /ve/bin/pip download sockjs-tornado

→ `AttributeError: '_NamespacePath' object has no attribute 'sort'`

More relevantly, it breaks pip-compile in the same way.  To fix this,
we need to force setuptools to be reinstalled, even if we’re asking
for the same version.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-23 13:23:58 -07:00
Anders Kaseorg
d368aaad10 requirements: Generate pip.txt from pip.in like the other *.txt files.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-23 13:23:58 -07:00
Anders Kaseorg
ec9bf6576a requirements: Remove unnecessary version bounds from *.in.
This makes no changes to the locked versions in *.txt, but it reduces
duplicate information and gives us sane workflows for

* upgrading packages: remove some or all lines from *.txt and re-run
  `update-locked-requirements`;
* marking packages as intentionally held back: add a version bound
  to *.in with an explanatory comment.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-23 13:23:58 -07:00
Anders Kaseorg
75e585b208 docs: Suggest systemctl unmask docker if Docker isn’t running.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-23 12:52:07 -07:00
Anders Kaseorg
9339402541 messages: Sort IDs in the by_streams query deterministically.
This fixes a test flake after upgrading SQLAlchemy.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-23 11:56:52 -07:00
Anders Kaseorg
cb0652071f zulip_otp_required: Avoid _user_is_authenticated compatibility shim.
This private shim is not needed with Django ≥ 1.10, and was removed
from django-otp 0.7.0.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-23 11:56:52 -07:00
Anders Kaseorg
2573ecb7d1 requirements: Bump thumbor Django to match main Django.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-23 11:56:36 -07:00
Anders Kaseorg
92b42573fb requirements: Compile thumbor requirements with pip-tools on Python 2.
The reason that `pip-tools` running on Python 3 didn’t detect the
right requirements for `thumbor` on Python 2 is simply that some of
them are conditional on the Python version.

As for the requirements that had been manually added as a workaround:
`backports-abc` and `singledispatch` are now correctly detected, while
`backports.ssl-match-hostname` was vendored into `urllib3` some time
ago and `certifi` is no longer necessary.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-23 11:56:36 -07:00
Tim Abbott
3c0f54e242 docs: Update documented string_id for zulipinternal realm.
See 9b2b24c617 for the update of the
string_id.
2019-09-23 11:51:10 -07:00
Tim Abbott
e4dee9532c nagios: Update configuration for user_activity worker change.
Since LoopQueueProcessingWorker jobs cannot be monitored by checking
for connected consumers (since they poll, rather than consuming as
events arrive), they can't be monitored with check_consumers.  It's
OK, because that monitoring was redundant with monitoring for
potential growth in their queue that we have as well.

Also clean up the block comments for the two other similar queue
procesors.
2019-09-23 11:49:46 -07:00
Rishi Gupta
3646936cfb integrations: Advertise our native Zapier integration. 2019-09-22 18:47:42 -07:00
Anders Kaseorg
8d91bebf95 restart-server: Warn if the shell’s PWD goes through an updated symlink.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-21 12:02:15 -07:00
Ray Kraesig
a4ababb9a4 vagrant docs: specify restart command
Add a specific command to restart Vagrant to adopt the new
configuration.

(When naïvely using only `vagrant halt` + `vagrant up --provision`,
external devices remained unable to connect; per `netstat -nltp`, the
host IP of forwarded ports remained `127.0.0.1`.)
2019-09-21 11:54:14 -07:00
Tim Abbott
767a1b9f0c user_activity: Skip an unnecessary database query for new entries.
While new entries are an overwhelming minority of calls to this
function, it's still worth having a clean implementation.
2019-09-21 11:49:43 -07:00
Tim Abbott
1c73ce2450 user_activity: Use LoopQueueProcessingWorker strategy.
This should dramatically improve the queue processor's performance in
cases where there's a very high volume of requests on a given endpoint
by a given user, as described in the new docstring.

Until we test this more broadly in production, we won't know if this
is a full solution to the problem, but I think it's likely.  We've
never seen the UserActivityInterval worker end up backlogged without a
total queue processor outage, and it should have a similar workload.

Fixes #13180.
2019-09-21 11:48:24 -07:00
Tim Abbott
52a2b2c6a8 tests: Move user_activity test next to slow_query test.
This will conveniently put all the LoopQueueProcessingWorkers
together.
2019-09-21 11:47:14 -07:00
Tim Abbott
f0d8951035 do_update_user_activity: Refactor to support passing a count.
We'll use this in upcoming commits.
2019-09-21 11:47:14 -07:00
Tim Abbott
5c960b3e0f user_activity: Make the queue processor a bit more efficient.
We don't actually need to go to the memcached (falling back to the
database) to fetch either user or client objects on every event.  For
user objects, we actually can just pass through the user ID
transparently; for client objects, we can use an in-process cache,
since the mapping of string to ID never changes.
2019-09-21 11:47:14 -07:00
Mateusz Mandera
3271235200 email_mirror: Ignore missed message email if the user isn't active. 2019-09-20 17:58:10 -07:00
Mateusz Mandera
21459ed193 test_email_mirror: Test missed message emails in a single ZulipTestCase.
With the way these tests are, it's unnecessary to have 3 separate
classes, and it makes it confusing to decide where to add a potential
additional mm email test.
2019-09-20 17:58:10 -07:00
Mateusz Mandera
371a7768ad backends: Extract useful is_user_active function.
This logic can be useful elsewhere, for checking whether user_profile is
active.
2019-09-20 17:58:10 -07:00
Tim Abbott
c19b7ec89d test_events: Remove unnecessary check_none validator.
This can be done cleanly using the existing equals(None).
2019-09-20 11:51:50 -07:00
Anders Kaseorg
6b09e690f1 page_params: Throw an error if params are missing.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-20 10:34:44 -07:00
Anders Kaseorg
dea6889956 templates: Make the Loading… message more robust.
Don’t hide it until both CSS and JS have loaded.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-20 10:34:44 -07:00
Anders Kaseorg
46e562f990 bootstrap: Change tooltip html default to false.
Bootstrap v2.2.0^2~40^2~6 changes this default to false, so this is a
prerequisite to upgrading Bootstrap, and it’s also safer.

This closes an HTML injection path via user full names in the emoji
reaction tooltip.  It doesn’t appear to be exploitable for cross-site
scripting because we disallow `>` in full names, and the code happens
to be written such that the next `>` is in a different parser
invocation.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-19 20:53:10 -07:00
Anders Kaseorg
cf5a00d94b bulk_get_subscriber_user_ids: Sort each user list by ID.
This simple backwards-compatible change saves approximately 12% in the
compressed size of the chat.zulip.org page_params.  We can do much,
much better by changing the format, but this seems like a good
intermediate step.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-19 20:17:43 -07:00
Anders Kaseorg
2e1494bdbd setup-apt-repo: Add ca-certificates to pre_setup_deps.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-19 20:15:43 -07:00
Rishi Gupta
74d5d3db26 integration docs: Remove angle brackets from trello bash command.
If you accidentally leave them in, you'll get a confusing error message.
2019-09-19 17:48:16 -07:00
Tim Abbott
a352f2e10d test_counts: Remove custom user creation code.
Like the last commit, this avoids us needing to update this random
test class in the analytics subsystem when we adjust the UserProfile
model.
2019-09-19 14:31:58 -07:00
Tim Abbott
aae610f65c analytics: Fix creation of shylock user.
Previously, the shylock user was created using a bad duplicate of our
standard create_user code path; we fix this by just calling into that
code path.
2019-09-19 14:16:38 -07:00
Yago González
e1fbf6dddb actions: Make do_regenerate_api_key return the new key.
This way, the new API key can be fetched without needing to read it from
the UserProfile object.
2019-09-19 13:03:56 -07:00
Anders Kaseorg
fbc2de157e templates: Move page_params to a <div> at the bottom of <body>.
In a gigantic realm where we send several MB of `page_params`, it’s
slightly better to have the rest of the `<body>` available to the
browser earlier, so it can show the “Loading…” spinner and start
fetching subresources.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-19 12:38:10 -07:00
Mateusz Mandera
935702b6ec auth: Rename any_oauth_backend_enabled to any_social_backend_enabled.
any_oauth_backend_enabled is all about whether we will have extra
buttons on the login/register pages for logging in with some non-native
backends (like Github, Google etc.). And this isn't about specifically
oauth backends, but generally "social" backends - that may not rely
specifically rely on Oauth. This will have more concrete relevance when
SAML authentication is added - which will be a "social" backend,
requiring an additional button, but not Oauth-based.
2019-09-19 12:35:27 -07:00
Mateusz Mandera
050f814e47 auth: Clean up SOCIAL_AUTH_BACKENDS / OAUTH_BACKEND_NAMES lists.
SOCIAL_AUTH_BACKEND / OAUTH_BACKEND_NAMES are currently the same
backends. All Oauth backends are social, and all social are oauth.
So we get rid of OAUTH_BACKEND_NAMES and use only SOCIAL_AUTH_BACKENDS.
2019-09-19 12:35:27 -07:00
Rohitt Vashishtha
6a08efc065 typeahead: Precompile regexes for removing diacritics.
Precompiling regexes gives a performance increase of around 10-15%
based on tests. See https://jsperf.com/typeahead-regex. This stacks
up when we have a lot of users in an organisation.
2019-09-19 12:27:01 -07:00
Anders Kaseorg
b72bb8171b nginx: Add CORS, HSTS, and X-Frame-Options headers to error responses.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-19 12:10:18 -07:00
Anders Kaseorg
990f08da90 dependencies: Upgrade nearly all JavaScript dependencies to latest.
Keep katex at 0.10.x, mini-css-extract-plugin at 0.6.x
(https://github.com/webpack-contrib/mini-css-extract-plugin/issues/417),
and typescript-eslint at 1.x for now.  Upgrade everything else.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-19 12:08:52 -07:00
Anders Kaseorg
fc0554fcb8 emoji: Finish removing leftover code from banned Emoji sets.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-19 12:08:52 -07:00
clarammdantas
cf5d3a3ef3 left sidebar: Fix bot availability status in "private messages".
This changes the availability icon for bot users to user_circle_green;
previously it was accidentally defaulting to user_circle_empty, making
it appear that bots were never available.

Fixes #13149.
2019-09-18 17:40:25 -07:00