Commit Graph

7343 Commits

Author SHA1 Message Date
sahil839
05aff3f271 api: Add "ignored_parameters_unsupported" to response of '/settings'.
We add "ignored_parameters_unsupported" field to the response object
of 'PATCH /settings' endpoint. This will contain the parameters
passed to the endpoint which are not changed by the endpoint and are
ignored.

This will help in removing the other fields like "full_name" from
response which was essentially present to specify that only these
fields were updated by the endpoint and rest were ignored.

We will also change other endpoints to follow this in future.
2021-07-16 16:10:15 -07:00
Anders Kaseorg
fb3ddf50d4 python: Fix mypy no_implicit_reexport errors.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-16 14:02:31 -07:00
Anders Kaseorg
1a985911ef test_classes: Use mock.patch in tornado_redirected_to_list.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-16 14:02:31 -07:00
Anders Kaseorg
1ae56e466b cache: Fix typing for post_save and post_delete flush handlers.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-16 13:14:04 -07:00
PIG208
49eab4efef Revert "request: Refactor to record rate limit data using ZulipRequestNotes."
This reverts commit 3f9a5e1e17.
2021-07-16 09:01:20 -07:00
Abhijeet Prasad Bodas
2545f00ad8 do_update_embedded_data: Remove unnecessary transaction.atomic.
There isn't any attachments code involved here.
This was added in c93f1d4eda, probably accidentally.
2021-07-16 09:00:02 -07:00
akshatdalton
6b5812082e markdown: Fix shebang line eliminating behaviour of Codehilite.
See the block comment explaining the motivation for this change, but
basically, the shebang feature of Python-Markdown's Codehilite
extension could be really confusing and is not part of the CommonMark
standard.

1. https://python-markdown.github.io/extensions/code_hilite/#shebang-no-path
2. eacff473a2/markdown/extensions/codehilite.py (L164-L180)

Fixes: #18591.
2021-07-15 15:18:33 -07:00
Anders Kaseorg
7ccefa5242 push_notifications: Fix APNs message generation.
This emulates the previous PyAPNs2 behavior of moving the contents of
the ‘custom’ key to top level.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-15 13:49:56 -07:00
Eeshan Garg
faa695e86d registration: Collect organization type on sign-up. 2021-07-15 09:58:27 -07:00
sahil839
61acf68aa4 emoji: Allow emoji authors to delete the emoji always.
Previously, non-admin emoji authors were allowed to
delete the emoji only if add_emoji_by_admins_only
was false. But, as add_emoji_by_admins_only setting
is for who can add emoji and not delete emojis, it
should not affect the behavior of deleting emojis
and users should always be allowed to delete the
emojis which. they added themselves
2021-07-14 14:57:08 -07:00
sahil839
50240ca71b models: Use COMMON_POLICY_TYPES for user_group_edit_policy.
This commit adds moderators and full members options for
user_group_edit_policy by using COMMON_POLICY_TYPES.

Moderators do not require to be a member of user group in
order to edit or remove the user group if they are allowed
to do so according to user_group_edit_policy.
But full members need to be a member of user group to edit
or remove the user group.
2021-07-14 14:38:44 -07:00
sahil839
93a1479286 user_group: Change error message to "Insufficient permission".
There is no need to have a error message which specifies the
roles having permission to edit user-groups, we can simply
have error message as "Insufficient permission" as we already
show the roles having permission clearly in UI.
2021-07-14 14:38:43 -07:00
sahil839
d7dfe80454 models: Add RealmUserDefault table for realm-level default of settings.
This table will be used to store the realm-level default of display
and notification settings for new users.
2021-07-14 14:35:04 -07:00
PIG208
c03b9c95ad request: Store client information using ZulipRequestNotes.
This concludes the HttpRequest migration to eliminate arbitrary
attributes (except private ones that are belong to django) attached
to the request object during runtime and migrated them to a
separate data structure dedicated for the purpose of adding
information (so called notes) to a HttpRequest.
2021-07-14 12:01:07 -07:00
PIG208
26875cdf0b tests: Migrate mocked Request to HostMockRequest.
This migrates some mocked Request class and mocked request achieved
with namedtuple in test_decorators and test_mirror_users to use the
refactored HostMockRequest.

Since weakref cannot be used with namedtuple, this old way of mocking a
request object should be migrated to using HostRequestMock. Only after
this change we can extract client from the request object and store it
via ZulipRequestNotes.
2021-07-14 12:01:07 -07:00
PIG208
5167a93229 request: Move tornado_handler to ZulipRequestNotes. 2021-07-14 12:01:07 -07:00
PIG208
742c17399e request: Move miscellaneous attributes to ZulipRequestNotes.
This includes the migration of fields that require trivial changes
to be migrated to be stored with ZulipRequestNotes.

Specifically _requestor_for_logs, _set_language, _query, error_format,
placeholder_open_graph_description, saveed_response, which were all
previously set on the HttpRequest object at some point. This migration
allows them to be typed.
2021-07-14 12:01:07 -07:00
PIG208
3f9a5e1e17 request: Refactor to record rate limit data using ZulipRequestNotes.
We will no longer use the HttpRequest to store the rate limit data.
Using ZulipRequestNotes, we can access rate_limit and ratelimits_applied
with type hints support. We also save the process of initializing
ratelimits_applied by giving it a default value.
2021-07-14 12:01:07 -07:00
PIG208
da6e5ddcae request: Move log_data from HttpRequest to ZulipRequestNotes. 2021-07-14 12:01:05 -07:00
PIG208
75cde8ebcf request: Refactor HostRequestMock to subclass HttpRequest. 2021-07-14 11:52:43 -07:00
PIG208
03693cd27e request: Map HttpRequest to ZulipRequestNotes for typing.
We create a class called ZulipRequestNotes as a new home to all the
additional attributes that we add to the Django HttpRequest object.
This allows mypy to do the typecheck and also enforces type safety.

Most of the attributes are added in the middleware, and thus it is
generally safe to assert that they are not None in a code path that
goes through the middleware. The caller is obligated to do manual
the type check otherwise.

This also resolves some cyclic dependencies that zerver.lib.request
have with zerver.lib.rate_limiter and zerver.tornado.handlers.
2021-07-14 11:52:42 -07:00
akshatdalton
f5c4d51ed2 resolve topic: Add is:resolved search keyword/filtering support.
This commit adds the backend support for `is:resolved` search keyword.
In the next commit, I will add the frontend support for the same.
2021-07-13 23:18:41 -07:00
akshatdalton
7ec406f39d refactor: Extract RESOLVED_TOPIC_PREFIX in topic.py.
This is a prep commit for #18990.
2021-07-13 23:18:41 -07:00
Abhijeet Prasad Bodas
1709428cff models: Create MissedMessageEmailEntry table.
This will be used to store the missedmessage events received
during the waiting time for email notifications (which is currently
2 minutes, hardcoded).

The change in `test_retention` is because we've set `on_delete=CASCADE`
for the message field this table.
The new query is like so:
```
DELETE FROM "zerver_missedmessageemailentry"
    WHERE "zerver_missedmessageemailentry"."message_id" IN (
        1545, 1546, 1547, 1548, 1549, 1550, 1551, 1552, 1553
        )
```
2021-07-13 17:21:37 -07:00
Abhijeet Prasad Bodas
c3319a5231 notification_data: Create common source for trigger strings.
This reduces loose strings in the codebase, and allows us to not worry
about the exact naming (`stream_email_enabled` or `stream_emails_enabled`?)
and tense (`mentioned` or `mention`?).

Ideally this new class should have been in `lib/notification_data.py`,
which is our file for things like this. But, the next commit requires
using this data in `models.py`, and importing from `notification_data.py`
to `models.py` causes recursive imports.
2021-07-13 17:16:32 -07:00
Alex Vandiver
0b7cff2f22 push_notifications: Hide "error" level messages from aioapns.
Work around Fatal1ty/aioapns#15, by silencing error-level logging from
the aioapns logger.  We deal with the results of failed
send_notification calls by examining the `result.description` and
handling them; the extra logging message merely clutters the Sentry
logs.
2021-07-13 09:28:05 -07:00
Anders Kaseorg
8486499314 fenced_code: Fix processor type annotation.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-09 17:10:31 -07:00
Tim Abbott
bc42ba87d4 message: Add types for RawUnreadMessagesResult keys.
This improves the readability for this important piece of code.
2021-07-09 11:34:07 -07:00
m-e-l-u-h-a-n
781179c56a subs: Rename subs.js to stream_settings_ui.js.
We use subs as a common variable name for a collection of stream
data structure used in settings, in lot of modules. So this
rename clears a bunch of related shadowed variables.
2021-07-09 09:38:58 -07:00
Tim Abbott
c804ab27d5 actions: Rename do_activate_user to do_activate_mirror_dummy_user.
This function had a confusing name, which could result in someone
using it unintentionally when they meant do_reactivate_user.

We also add docstrings for both functions.
2021-07-08 17:33:17 -07:00
Mateusz Mandera
b9056d193d rate_limit: Implement IP-based rate limiting.
If the user is logged in, we'll stick to rate limiting by the
UserProfile. In case of requests without authentication, we'll apply the
same limits but to the IP address.
2021-07-08 15:46:52 -07:00
Gaurav Pandey
d2074fc10c events: Remove sender_id from delete_message event.
Remove `sender_id` as part of responses from `/events`
api for `delete_message` event when `message_type` is
`private`.
2021-07-08 13:40:32 -07:00
Gaurav Pandey
bc131c1636 events: Remove recipient_id from delete_message event.
Remove `recipient_id` from `delete_message` event response
when `message_type` is `private`. API changelog updated in
next commit.
2021-07-08 13:36:45 -07:00
sahil839
a535ec6262 settings: Remove timezone from UserProfile.property_types.
We remove timezone setting from UserProfile.property_types
so that we can directly use UserProfile.property_types for
implementation of realm-default values of various user
settings.
2021-07-08 12:52:29 -07:00
aryanshridhar
9049fb3bd4 users: Remove redundant compute_show_invites function.
This commits removes the redundant `compute_show_invites` function
which computes the `show_invites` page parameter in `lib/users.py`.

It is so because, commit 13399833b0 removed
the `show_invites` context variable passed in index.html.
Hence, the `show_invites` page_param key is no
longer required to compute in backend as it can be switched with
`settings_data.user_can_invite_others_to_realm()` in the frontend.

This commits also removes the `test_compute*` tests in
`test_home` that concerned with the `show_invites` page parameter
as they are no longer required.
2021-07-08 12:37:47 -07:00
Abhijeet Prasad Bodas
bf15c0235a notifications: Remove unused fields in queue events.
* `stream_name`: This field is actually redundant. The email/push
  notifications handlers don't use that field from the dict, and they
  anyways query for the message, so we're safe in deleting this field,
  even if in the future we end up needing the stream name.

* `timestamp`: This is totally unused by the email/push notification
  handlers, and aren't sent to push clients either.

* `type` is used only for the push notifications handler, since only
  push notifications can be revoked, so we move them to only run there.
2021-07-08 11:22:45 -07:00
Abhijeet Prasad Bodas
76dc504f7e push_notifications: Remove outdated comments.
We solely rely on the logic here for filtering with the `read` flag
since 10dd5f784b.
2021-07-08 11:14:08 -07:00
Tim Abbott
af47fa705e exceptions: Use HTTP 401 code for authentication errors. 2021-07-08 10:33:08 -07:00
Abhijeet Prasad Bodas
4f9c7cae0a push_notifications: Send mentioned user group ID and name in payload. 2021-07-08 10:19:43 -07:00
Abhijeet Prasad Bodas
9bd8fe01fc android push notifications: Display mentioned user group name.
Followup to 83399e2e72.
2021-07-08 10:03:07 -07:00
Abhijeet Prasad Bodas
4d24499317 android notifications: Differentiate personal vs wildcard mentions.
The code to also notify for wildcard mentions was added in
0ed0bb6828.

But that showed the same text for both the cases. This commit fixes
that.

This is more of change for correctness. The mobile app currently does
not rely on this text for notifications, but constructs the text by
itself from the data in the payload.

This also fixes the "stream_push_notify" case to consistently show
a `#` before the stream name.
2021-07-08 10:03:07 -07:00
Abhijeet Prasad Bodas
ce6f6a3829 push_notifications: Pre-calculate mentioned_user_group_name.
Prep change for showing the mentioned user group name in Android
notifications also. This will avoid doing the user group fetch twice.
2021-07-08 10:03:07 -07:00
akshatdalton
db1cf3b521 refactor: Add class ZulipBaseHandler to de-duplicate code. 2021-07-07 17:53:22 -07:00
Anders Kaseorg
98194b9316 blueslip: Remove SAVE_FRONTEND_STACKTRACES setting.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-07 09:26:02 -07:00
Anders Kaseorg
07fef56c74 logging_handlers: Remove STAGING_ERROR_NOTIFICATIONS setting.
Running notify_server_error directly from the logging handler can lead
to database queries running in a random context.  Among the many
potential problems that could cause, one actual problem is a
SynchronousOnlyOperation exception when running in an asyncio event
loop.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-07 09:26:02 -07:00
Vishnu KS
acffc0ae0a populate_db: Use do_create_realm for creating zephyr realm. 2021-07-06 17:22:00 -07:00
Vishnu KS
b9066886d9 cache: Properly flush stream from cache after the stream is deleted.
The previous logic was incorrect and was not flushing the stream from
cache after deletion.

```

stream = get_realm_stream("Verona", realm.id)
stream.delete()
get_realm_stream("Verona", realm.id)
```

In the above example, the last line of code would have returned
the stream from cache instead of throwing a Stream.DoesNotExist
error. This is fixed in the commit.

I have verified that this commit indeed fix the issue by verifying
that calling get_realm_stream again after deleting the stream
results in Stream.DoesNotExist error.
2021-07-06 17:21:59 -07:00
PIG208
8b9011dff8 json_error: Completely remove json_error.
This completes the migration from `return json_error` to
`raise JsonableError`.
2021-07-06 15:34:33 -07:00
Tim Abbott
dbf886dfce exceptions: Deduplicate common authentication errors. 2021-07-06 15:28:38 -07:00
Tim Abbott
331f34cc1f auth: Use standard classes for API key fetch exceptions.
This lets us reuse a bunch of code and error handling logic, resulting
in a more sensible and consistent API.
2021-07-06 15:28:38 -07:00