Commit Graph

3601 Commits

Author SHA1 Message Date
Tim Abbott
31afa36d7b unminify: Clean up unnecessary repetition of webpack:/// URLs.
This takes stacktrace lines that used to look like this:

n@https://chat.zulip.org/static/webpack-bundles/app.2385793af60f0b082ee9.js:1:12680
       = webpack:///./static/js/blueslip.js line 241 column 1
dispatch@https://chat.zulip.org/static/webpack-bundles/app.2385793af60f0b082ee9.js:52:37878
       = webpack:////srv/zulip-npm-cache/8ea4cd291dd23441aec0f298b77b6ddc0d0a7a56/node_modules/jquery/dist/jquery.js line 5182 column 1

to have the even-numbered lines look like this:
       = ./static/js/blueslip.js line 241 column 1
dispatch@https://chat.zulip.org/static/webpack-bundles/app.2385793af60f0b082ee9.js:52:37878
       = /srv/zulip-npm-cache/8ea4cd291dd23441aec0f298b77b6ddc0d0a7a56/node_modules/jquery/dist/jquery.js line 5182 column 1
2018-08-03 15:56:40 -07:00
Yago González
936d8c32c0 test-api: Fix typo in message event types.
The event type for messages is "message", not "messages" as the test
(and hence the API docs Python example) reflected.
2018-08-02 15:54:41 -07:00
Yago González
2b8bb2ffc6 api docs: Migrate DELETE /events to OpenAPI. 2018-08-02 15:54:41 -07:00
Yago González
5631645d9f api docs: Migrate /register to OpenAPI. 2018-08-02 15:54:41 -07:00
Yago González
7d8e058e39 api docs: Implement an exception list for schema validation. 2018-08-02 15:54:41 -07:00
Tim Abbott
e7c7211c30 mypy: Fix type of messages in do_update_message_flags.
Ever since we moved the stream/everything cases to separate functions,
the messages argument has actually been required.
2018-08-01 17:37:16 -07:00
Tim Abbott
23246ff816 do_update_pointer: Switch to using where_unread().
This produces a more efficient database query (just because postgres
doesn't use the right index by default the other way).
2018-08-01 16:51:56 -07:00
Tim Abbott
c775be8ea4 do_mark_stream_messages_as_read: Accept a Client object.
We also fix an incorrect Optional in the type annotations.
2018-08-01 16:49:57 -07:00
Tim Abbott
6e55342e21 bulk_remove_subscriptions: Pass client object in.
We need the client object to pass on to do_mark_stream_as_read.
2018-08-01 16:48:31 -07:00
Tim Abbott
c60f197fde do_update_message_flags: Accept a Client object.
This is important for upcoming logging changes.
2018-08-01 16:40:58 -07:00
Tim Abbott
0e44010976 do_mark_all_as_read: Accept a client object.
This is needed for upcoming logging changes.
2018-08-01 16:40:15 -07:00
Tim Abbott
5f0519dfb4 do_update_pointer: Pass client object from callers.
We also fix an unused import.

This is needed for upcoming logging changes.
2018-08-01 16:40:15 -07:00
Tim Abbott
54d558b128 management: Add library for getting a client object.
This is to be used in some analytics features we're adding in the near
future.
2018-08-01 16:40:15 -07:00
Tim Abbott
e9f4d9db2b push_notifications: Fix interface for handle_remove_push_notification.
This really should just accept a message ID.
2018-08-01 16:36:42 -07:00
Kunal Gupta
bc43eefbfb notifications: Add function for cancelling GCM notifications.
This adds a new function called handle_remove_push_notification in
zerver/lib/push_notifications.py which requires user_profile id and
the message id which has to be removed in the function.

For now, the function only supports GCM (and is mostly there for
prototyping).

The payload which is being delivered needs to contain the narrow
information and the content of the message.
2018-08-01 15:59:04 -07:00
Tim Abbott
5f8d193bb7 notifications: Include realm_uri in push notifications.
This should make it much simpler for the mobile apps to line up the
data from server_settings against the data in the notifications.

Addresses part of #10094.
2018-08-01 15:46:15 -07:00
Tim Abbott
58ee3fa8c4 page_params: Include avatars and similar data in cross-realm bots.
This ensures that the format of this data structures matches that for
in-realm bots in the main users data structure (including avatars,
etc.).

Fixes #10138.
2018-08-01 15:09:11 -07:00
Tim Abbott
ccba1e7c0e cache: Limit which realms we access when filling caches.
For realms that don't have any presence-active users, we know for a
fact that there aren't any active clients that will be reloading just
after the server restarts, so we can skip filling the cache with data
related to that realm.

For zulipchat.com, this results in a significant performance
optimization for the recipient and stream caches, and a moderate
performance improvement for the user caches as well.
2018-08-01 14:22:49 -07:00
Tim Abbott
e6abc6e0bd cache: Only ensure we fill recipient caches for Recipient.STREAM.
Private messages make up the bulk of Recipient objects.  While private
messages are ~50% of messages, if you weight by messages received
(which is what is important for message-loading performance), it's
pretty strongly balanced towards stream messages.
2018-08-01 13:09:59 -07:00
Tim Abbott
463a348a86 cache: Don't pre-populate low-traffic user caches.
We only really need to pre-populate the caches used for (1) mobile
authentication and (2) most other user lookups.
2018-08-01 13:09:34 -07:00
Tim Abbott
c42302e47b restart-server: Optimize prepopulating user cache.
We don't need to include long-term idle or other inactive users here,
since fetching them consumed to vast majority of the time.

(On chat.zulip.org, this decreased the runtime for populating the user
cache by 5x, removing only users we're unlikely to need to access).
2018-08-01 12:54:06 -07:00
Tim Abbott
f10e006135 message: Bump MAX_UNREAD_MESSAGES to 50000.
This doesn't seem to have a huge performance downside (less than 1s
extra time for loading / on chat.zulip.org), and it means the
possibility of users having so many unreads that we get weird/buggy
behavior is much more unlikely to exist.

We'll still want a better experience for users who somehow go over
this limit, but it can be pretty firmly "you need to go mark some
things as read".
2018-08-01 12:02:54 -07:00
Rhea Parekh
ee37866687 import: Add gitter import file in zerver/data_import directory. 2018-08-01 11:52:14 -07:00
Rhea Parekh
b8e1e8b31d import: Add slack import files in zerver/data_import directory. 2018-08-01 11:52:14 -07:00
Vishnu Ks
6b3706494c notifications: Pass realm_creation argument to enqueue_welcome_emails. 2018-08-01 11:29:34 -07:00
Roman Godov
5e70577f84 models: Rename Realm.show_digest_email field.
This renames Realm.show_digest_email field to
digest_emails_enabled, for greater clarity as to what it does
just from seeing the setting name, without having to look it up.

Fixes part of #10042.
2018-08-01 11:05:58 -07:00
Tim Abbott
6f7e12ea19 docs: Add subsystem documentation for caching. 2018-07-31 17:00:45 -07:00
Tim Abbott
1b2a26ca83 events: Fix missing empty custom profile data dict for new users.
We were getting event-handling exceptions in JS in production if a new
user was created and then went and set a custom profile field, because
there was no `.profile_data` on their user object.  We were able to
trace the issue down to the fact that our events didn't include that
field when creating a new user.
2018-07-31 11:08:11 -07:00
Roman Godov
c0806917ec models: Rename Realm.restricted_to_domain field.
This renames Realm.restricted_to_domain field to
emails_restricted_to_domains, for greater clarity as to what it does
just from seeing the setting name, without having to look it up.

Fixes part of #10042.
2018-07-31 09:28:33 -07:00
Cynthia Lin
29442ffb93 zcommand: Add light/dark mode command aliases for day/night mode.
Fixes #10095.
2018-07-31 07:12:31 -04:00
Tim Abbott
e04156eef3 export: Fix error messages for stream list mismatches.
The previous error messages for this were written for a tool only to
be used by a couple people, and didn't make clear what potential
causes were.  Tweak these to provide greater clarity about what's
going on.

The main cause of these errors appearing in practice was fixed in
7ea5987e5d, but nothing strongly
prevents a similar issue from being introduced in the future.

Fixes #10078.
2018-07-30 22:32:26 -07:00
Tim Abbott
6317064210 unminify: Fix source map extraction for hash-named files.
Apparently, our old unminify logic relied on the fact that the
filenames displayed in tracebacks were of the form "app.js" (and the
`app.js` copy of the source map in the appropriate
`/home/zulip/deployments/`).  The correct behavior is to just look up
the source map for the appropriate hash-named
`app.a40806b10565c1dee5bf.js` type file.

We fix this with a few small tweaks to the regular expressions.  I
wish this file had reasonable unit tests.
2018-07-30 22:09:37 -07:00
Tim Abbott
aa5959396d docs: Add some basic subsystem documentation for thumbnailing. 2018-07-30 13:20:54 -07:00
Aditya Bansal
77651ece39 thumbnails: Rename size value 'original' to 'full'. 2018-07-30 13:00:23 -07:00
Aditya Bansal
5b5d8bb310 thumbnails: Rename data-original to data-src-fullsize. 2018-07-30 13:00:23 -07:00
Tim Abbott
02ae71f27f api: Stop using API keys for Django->Tornado authentication.
As part of our effort to change the data model away from each user
having a single API key, we're eliminating the couple requests that
were made from Django to Tornado (as part of a /register or home
request) where we used the user's API key grabbed from the database
for authentication.

Instead, we use the (already existing) internal_notify_view
authentication mechanism, which uses the SHARED_SECRET setting for
security, for these requests, and just fetch the user object using
get_user_profile_by_id directly.

Tweaked by Yago to include the new /api/v1/events/internal endpoint in
the exempt_patterns list in test_helpers, since it's an endpoint we call
through Tornado. Also added a couple missing return type annotations.
2018-07-30 12:28:31 -07:00
Shubham Dhama
499a70b01e notifications: Fix leaking of private stream msgs sent before subscribing.
Fixes: #9834.
2018-07-28 15:13:08 -07:00
Shubham Dhama
c5d8fdf68c message: Add function to check message access in bulk.
This effectively just calls access_message in a loop.
2018-07-28 15:12:55 -07:00
Tim Abbott
6bbffe0e2e notifications: Extract zerver/lib/url_encoding.py.
Extracting this helper library will help us avoid an import loop
between notifications.py and message.py (with bugdown in between).

But in addition to that, it's a more natural model, since some of the
uses for these functions weren't part of the notifications code
anyway.
2018-07-28 15:12:55 -07:00
Shubham Dhama
605a6ed3d8 message: Extract function for message access by a user. 2018-07-28 14:58:36 -07:00
Shubham Dhama
9cf67a5e71 models: Add function to get UserMessage from message id. 2018-07-28 14:58:36 -07:00
Tim Abbott
38dd9e49de import_realm: Add comments for update_model_ids.
This basically explains why in these cases we delay doing
bulk_import_model.
2018-07-26 16:13:14 -07:00
Rhea Parekh
fca6bc91aa import: Add 'get_db_table' function.
Implement this function in 'bulk_import_model'
and 'update_model_ids'.

This lets us save on redundant-feeling arguments in these
frequently-called helper functions.
2018-07-26 16:07:31 -07:00
Roman Godov
064fca5b76 import: Fix use of unused Subscription.notifications.
This code should have been interacting with the specific sub-fields
for notifications.
2018-07-26 15:53:29 -07:00
Yago González
db81726bcb api docs: Use method from our Python bindings in the example.
One of the code examples for GET /users was using the raw call_endpoint
method from our Python bindings, rather than get_members, which has been
specifically designed to interact with this endpoint.

Now all the examples here use the appropiate method.
2018-07-26 15:34:31 -07:00
Yago González
40a21f6077 api docs: Migrate GET /users to OpenAPI. 2018-07-26 15:34:31 -07:00
Yago González
7c50f6cdd4 api docs: Migrate POST /messages/render to OpenAPI. 2018-07-26 15:34:31 -07:00
Yago González
57c2d8c72a api docs: Migrate GET /get_stream_id to OpenAPI. 2018-07-26 15:34:31 -07:00
Yago González
679319a735 api docs: Migrate GET /streams to OpenAPI. 2018-07-26 15:34:31 -07:00
Yago González
95ce311686 api docs: Display the default value of parameters when present.
Whenever a parameter for an endpoint in our REST API has a default
value, it is displayed under the "Description" section of the
arguments table in the docs.

This way, we don't need to explicitly indicate the default values in the
description, thus avoiding duplicate information in the OpenAPI source.
2018-07-26 15:34:31 -07:00