Commit Graph

33972 Commits

Author SHA1 Message Date
Mateusz Mandera
5d2befdc54 send_to_email_mirror: Fix loop setting recipient-like headers.
return in that loop was a bug, which would lead to the To: header not
being set even though data['recipient'] = str(message['To']) is being
run next, thus requiring the header. We can remove the return
statement and now the loop will overwrite all the potentially
troublesome headers.
2020-02-19 12:08:55 -08:00
Mateusz Mandera
cc8b83b261 email_mirror: Insert a new line before attachment links. 2020-02-19 12:08:55 -08:00
Mateusz Mandera
ac8f4aaa93 email_mirror: Check address usability in get_missed_message_address. 2020-02-19 12:08:55 -08:00
Mateusz Mandera
843c148c59 email_mirror: Give extract_and_validate a more descriptive name. 2020-02-19 12:08:55 -08:00
Mateusz Mandera
d39bcf2264 email_mirror: Reuse exception messages in mirror_email_message. 2020-02-19 12:08:55 -08:00
Tim Abbott
ce64a6b163 default stream groups: Fix broken registration UI.
The default stream groups feature (#6693) was never fully implemented;
this fixes a key detail (the registration UI being broken).
2020-02-19 12:08:55 -08:00
Tim Abbott
7875196783 default stream groups: Fix buggy LDAP behavior.
With LDAP authentication, we don't currently have a good way to
support the default stream groups feature.

The old behavior was just to assume a user select every default stream
group, which seems wrong; since we didn't prompt the user about these,
we should just ignore the feature.
2020-02-19 12:08:55 -08:00
Mateusz Mandera
56c1ad1a3d install: Don't create internal realm in the installation process. 2020-02-19 17:05:28 +01:00
Tim Abbott
d9aa4161f8 install: Remove references to "Zulip Voyager".
"Zulip Voyager" was a name invented during the Hack Week to open
source Zulip for what a single-system Zulip server might be called, as
a Star Trek pun on the code it was based on, "Zulip Enterprise".

At the time, we just needed a name quickly, but it was never a good
name, just a placeholder.  This removes that placeholder name from
much of the codebase.  A bit more work will be required to transition
the `zulip::voyager` Puppet class, as that has some migration work
involved.
2020-02-19 17:00:17 +01:00
Mateusz Mandera
728155afee server_initialization: Add server_initialized function. 2020-02-19 16:59:56 +01:00
Mateusz Mandera
660501c782 test_classes: Fix bug where UserProfile could be passed to client_post.
It would cause JSON overflow error while producing URL coverage report.
2020-02-19 16:59:14 +01:00
Mateusz Mandera
ad974c3ae3 initialize_voyager_db: Deduplicate create_internal_realm logic.
zerver.lib.server_initialization.create_internal has precisely the same
code (you can copy-and-paste swap them, with one level of indentation
adjustment, without generating any diff) so they can be trivially
deduplicated.
2020-02-19 16:57:44 +01:00
Mateusz Mandera
bc4029deae initialize_voyager_db: Deduplicate create_users.
zerver.lib.server_initialization.create_users has precisely the same
code (you can copy-and-paste swap them without generating any diff) so
they can be trivially deduplicated.
2020-02-19 16:54:39 +01:00
Mateusz Mandera
218ca61dd0 server_initialization: Rename some variables.
This makes the code of create_internal_realm identical to the
corresponding block in initialize_voyager_db.py.
2020-02-19 16:43:18 +01:00
Mateusz Mandera
3419908f39 initialize_voyager_db: Add comment above default client creation block. 2020-02-19 16:42:57 +01:00
Mateusz Mandera
af67990f14 server_initialization: Set internal bots owners to themselves. 2020-02-19 16:42:39 +01:00
Mateusz Mandera
e6cf30fc22 server_initialization: Remove unnecessary type annotation. 2020-02-19 16:42:15 +01:00
Mateusz Mandera
e2ccbe7c80 initialize_voyager_db: Add bot_owner argument to create_users.
This doesn't change any behavior, the purpose of this is to make the
function identical to what we have in server_initialization.py so that
it can be deduplicated in follow-up commits.
2020-02-19 16:41:56 +01:00
Mateusz Mandera
8b31387670 server_initialization: Use tos_version argument in create_users. 2020-02-19 16:41:31 +01:00
Mateusz Mandera
501eb09716 populate_db: Extract default client creation to server_initialization. 2020-02-19 16:25:30 +01:00
Mateusz Mandera
280d9db26d populate_db: Extract some functions to server_initialization.py. 2020-02-19 16:23:51 +01:00
Vishnu KS
cee6227f53 bots: Remove feedback cross realm bot.
This completes the remaining pieces of removing this missed in
d70e799466 (mostly in tests).
Backported to 2.1.x branch.
2020-02-19 16:21:02 +01:00
Mateusz Mandera
cae803e8a9 bots: Remove FEEDBACK_BOT implementation.
This legacy cross-realm bot hasn't been used in several years, as far
as I know.  If we wanted to re-introduce it, I'd want to implement it
as an embedded bot using those common APIs, rather than the totally
custom hacky code used for it that involves unnecessary queue workers
and similar details.
Backported to the 2.1.x branch.
2020-02-19 15:26:08 +01:00
Tim Abbott
ba598366e9 Release Zulip Server 2.1.2. 2.1.2 2020-01-16 12:26:14 -08:00
Steve Howell
d452ad31e0 server: Sort user_ids in recent PM conversations.
This change should prevent test flakes, plus
it's more deterministic behavior for clients,
who will generally comma-join the ids into
a key for their internal data structures.

I was able to verify test coverage on this
by making the sort reversed, which would
cause test_huddle_send_message_events to
fail.
2020-01-16 12:25:11 -08:00
Steve Howell
aed813f44c bug fix: Fix huddles in "Private Messages".
If two user_ids in a recent huddle have ids
that sort lexically differently than numerically,
such as 7 and 66, then we were creating two
different buckets in pm_conversations.

This regression was introduced in
263ac0eb45 on
November 21, 2019.
2020-01-16 12:25:11 -08:00
Steve Howell
71dae1b92a refactor: Have pm_conversations take user_ids.
Instead of having our callers pass in a possibly
non-canonical version of a user_ids_string, just
have them pass in a list.

The next commit will canonicalize the sort.
2020-01-16 12:25:11 -08:00
Steve Howell
629ec1aa8b tests: Use tricky server data in unit tests.
The server may send us ids in the order
[11, 2], instead of [2, 11].  We don't want
to rely on server behavior, regardless, for
the sort.

Our tests now show we process that data.

The current code is is still buggy and causes
us to show the same huddle two different times
for situations where the lexical sort doesn't
match the numerical sort.

This happens on czo often, where Tim is user
7, and his id sorts lexically after ids like
58, 622, 4444, etc.
2020-01-16 12:25:11 -08:00
Anders Kaseorg
87d60a1fff thumbnail: Tighten fix for CVE-2019-19775 open redirect.
Due to a known but unfixed bug in the Python standard library’s
urllib.parse module (CVE-2015-2104), a crafted URL could bypass the
validation in the previous patch and still achieve an open redirect.

https://bugs.python.org/issue23505

Switch to using django.utils.http.is_safe_url, which already contains
a workaround for this bug.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-01-16 11:41:12 -08:00
Tim Abbott
98eef54e4f i18n: Update translation data from Transifex. 2020-01-16 11:41:12 -08:00
Tim Abbott
235ba339d0 filter: Allow marking is:mentioned messages as read.
We may revisit this in the future, but similar to is:private, the
current Zulip user experience makes users expect that in the
is:mentioned view, they should really be able to mark messages as
read.

Further, the practice use case for not marking them as read is very
low, since it's rare for someone to have so many mentions that
revisiting the mentions view isn't sufficient to see everything that
needs their attention.
2020-01-16 11:15:46 -08:00
Tim Abbott
e5320cc1f6 filter: Add streams:public to sorted_term_types.
This is for consistency with in:, has:, and similar values where
there's a fixed set of RHS entries.
2020-01-16 11:15:04 -08:00
Rohitt Vashishtha
1d72ea2fd5 filter: Remove is_exactly().
Previously, is_exactly() had already been repalced with can_bucket_by().
This commit removes is_exactly() and replaces its usage in our tests
with can_bucket_by().
2020-01-16 11:14:44 -08:00
Steve Howell
c7948a7960 filter: Remove redundant is:private operators.
If we have a pm-with, then is:private is redundant
and just forces us to write confusing/verbose code
in various places.
2020-01-16 11:14:18 -08:00
Rohitt Vashishtha
04bb26be3a unreads: Remove is_reading_mode().
This was a part of an experiment we ran on chat.zulip.org in Jul 2018
and surrounding code that used it never got merged to master.

See: https://chat.zulip.org/#narrow/stream/2-general/topic/un-narrow.20view/near/609506
and c407ba5175.
2020-01-16 11:13:34 -08:00
Rohitt Vashishtha
7f45ca9b22 filter: Add 'in:*' to sorted_term_types.
This simplifies our handling of in-home and in-all cases in
can_mark_messages_read().
2020-01-16 11:13:16 -08:00
Steve Howell
1bedb965e9 refactor: Clean up can_mark_messages_read.
We now explicitly enumerate various cases, which
should make it easier to change this code.
2020-01-16 11:13:11 -08:00
Anders Kaseorg
bc752188e7 create-db.sql: Start by dropping the zulip database if needed.
At some point the PostgreSQL Docker image started creating the zulip
database for us, which caused our CREATE DATABASE to fail.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-01-15 18:12:35 -08:00
Anders Kaseorg
b0ea81fe16 create-db.sql: Handle exception if zulip user already exists.
Fixes #13530.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-01-15 18:12:31 -08:00
Anders Kaseorg
358ab821c4 generate_secrets: Enable redis authentication in production.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-01-15 14:03:15 -08:00
Anders Kaseorg
97322dd195 generate_secrets: Enable memcached authentication in production.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-01-15 14:03:15 -08:00
Anders Kaseorg
1ba48a04da settings: Support optional memcached authentication.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-01-15 14:03:15 -08:00
Tim Abbott
e8377b605f migrations: Fix zulipinternal migration corner case.
It's theoretically possible to have configured a Zulip server where
the system bots live in the same realm as normal users (and may have
in fact been the default in early Zulip releases?  Unclear.).  We
should handle these without the migration intended to clean up naming
for the system bot realm crashing.

Fixes #13660.
2020-01-15 14:02:07 -08:00
Tim Abbott
830f1e9f3f populate_db: Fix cache flushing when rebuilding test database.
This fixes a similar problem to the last commit; we don't use
memcached with the test database, so we don't need to flush memcached
when rebuilding it.

(And if we try, we'll get exceptions trying to access the relevant
settings).
2020-01-13 18:23:48 -08:00
Tim Abbott
037b87b580 populate_db: Fix handling of memcached flushing.
Our recent fixes to using the system's configured memcached settings
broke populate_db, because its hacky clear_database helper is called
with a hacked-up settings module.

We fix this by first moving this out-of-place code from models.py into
populate_db, and then saving the settings required to access memcached
so that we can use them in clear_database.

We also fix a mypy erorr in flush-memcached that matches the same
issue fixed in clear_database.
2020-01-13 18:23:44 -08:00
Anders Kaseorg
82a6e77301 flush-memcached: Use pylibmc.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-01-13 17:39:25 -08:00
Anders Kaseorg
9efb90510c clear_database: Respect MEMCACHED_LOCATION.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-01-13 17:39:22 -08:00
Anders Kaseorg
b255c8b8a6 puppet: Fix zuli-redis.conf path typo.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-01-13 17:37:28 -08:00
Anders Kaseorg
03e8e8be9d puppet: Delete legacy rediscleanup code.
It was added in commit 9afb1c7a71 from
before 1.4.0.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-01-13 17:37:25 -08:00
Ray Kraesig
2932d9cd28 docs: link to more-currently-maintained fork of GitX
The well-known rowanj/gitx repository hasn't been updated since 2014.
Preferentially direct new contributors to gitx/gitx instead.

(We retain the rowanj repo as a fallback, since it has precompiled
releases available.)
2020-01-13 17:36:47 -08:00