Commit Graph

3618 Commits

Author SHA1 Message Date
Steve Howell
2e42bdd7c3 Remove "/stats" command for now.
The "/stats" command doesn't actually do anything
interesting yet, and it also writes to the message
feed instead of replying directly to the user.

The history of this command was that it was
written during a PyCon sprint.  It was mainly intended
as an example for subsequent slash commands.  The
ones we built after "/stats" have sort of outgrown
"/stats" and don't follow the original structure
for "/stats".  (The "/day", "/ping", and "/settings"
commands were built shortly after.)j

We probably want to ressurect "/stats" fairly soon,
after figuring out some useful stats and refining
the UI.

As you can see from this commit, resurrecting the
code here shouldn't be too difficult, but it
may actually be pretty rare that we just translate
slash commands into fleshed out messages.
2018-08-08 16:49:27 -07:00
Yago González
298aa0fcbf mobile: Make otp_encrypt_api_key accept API keys.
Since otp_encrypt_api_key only encrypts API keys, it doesn't require
access to the full UserProfile object to work properly. Now the
parameter it accepts is just the API key.

This is preparatory refactoring for removing the api_key field on
UserProfile.
2018-08-08 16:45:40 -07:00
Yago González
6a192ac84c utils: Move random API key generator as generate_api_key.
random_api_key, the function we use to generate random tokens for API
keys, has been moved to zerver/lib/utils.py because it's used in more
parts of the codebase (apart from user creation), and having it in
zerver/lib/create_user.py was prone to cyclic dependencies.

The function has also been renamed to generate_api_key to have an
imperative name, that makes clearer what it does.
2018-08-08 16:45:25 -07:00
Yago González
f6219745de users: Get all API keys via wrapper method.
Now reading API keys from a user is done with the get_api_key wrapper
method, rather than directly fetching it from the user object.

Also, every place where an action should be done for each API key is now
using get_all_api_keys. This method returns for the moment a single-item
list, containing the specified user's API key.

This commit is the first step towards allowing users have multiple API
keys.
2018-08-08 16:35:17 -07:00
Tim Abbott
de45853c99 OpenAPI: Import yamole inside a function for performance.
This saves about 1% of the runtime of `manage.py showmigrations`
2018-08-08 14:19:42 -07:00
Tim Abbott
4d03c15848 url_preview: Don't import beautifulsoup at import time.
This is a small performance optimization to Django startup, in line
with other recent commits.
2018-08-08 14:19:42 -07:00
Tim Abbott
38afeb7ac2 bugdown: Lazily import python-twitter.
python-twitter was consuming a significant amount of import time.
However, this commit seems to not save any time at all, probably
because its recursive dependencies are imported elsewhere in Zulip.
2018-08-08 14:19:42 -07:00
Cynthia Lin
f68bb8132a zcommand: Fix broken switch mode message.
Instead of displaying the zcommand to switch to the other mode, the
message displayed the zcommand that the user had just entered.
2018-08-08 11:29:49 -07:00
Yago González
5566646c45 user_groups: Handle renaming to existing names.
Renaming a user group to a name shared by other group wasn't a scenario
handled by the backend, and the server errored whenever this was
attempted.

Now a json_error is returned, letting the user know that a user group
with that name already exists.
2018-08-08 11:03:47 -07:00
Yago González
8d0cf3ebe5 api docs: Migrate POST /user_uploads to OpenAPI. 2018-08-08 09:29:27 -07:00
Yago González
119b3c0bc4 api docs: Migrate DELETE /users/me/subscriptions to OpenAPI. 2018-08-08 09:29:27 -07:00
Yago González
14c9277095 api docs: Migrate POST /users to OpenAPI. 2018-08-08 09:29:27 -07:00
Yago González
e1662024d1 api docs: Load the OpenAPI file only when needed.
We found out in #9953 that, appparently, loading the OpenAPI file was
taking abut a 5% of the Zulip server startup time.

Since in many cases (especially in development) having the file loaded
won't be necessary at all, we read it on the first time data from the
OpenAPI spec is needed.

Tweaked by tabbott to add a test.
2018-08-08 09:00:28 -07:00
Yago González
25d2efb9ca api docs: Live reload the OpenAPI spec on update.
Automatically detect if the OpenAPI spec file has been modified since
the last time it was loaded into memory, and if it has, automatically
reload it to have the latest version.

This feature is designed with development environments in mind. The main
benefit is being able to see the changes made to the OpenAPI document
without needing to restart the development server, which is tedious and
slows the documentation workflow down.
2018-08-08 08:54:25 -07:00
Yashashvi Dave
290388e5e0 stream settings: Fix bug in UI when last user unsubscribe private stream.
When last user(only in case of admin) unsubscribe from private stream,
stream page doesn't get updated. Cause we delete the private stream
as soon as last user unsubscribe from stream.
So `sub` get undefined in frontend, cause that stream is deleted
before unsubscribe-user-from-stream event is received.

Fix this by changing order of events sent to frontend. Event
`subscription: remove` should be sent before `stream: delete` event
from backend.
2018-08-07 13:30:53 -07:00
Yashashvi Dave
6e136be975 default stream: Allows admins to remove any default stream.
This fixes a bug where administrators couldn't remove private
unsubscribed streams from the "default streams" list, because
access_stream_by_name didn't give them access to the stream object.
2018-08-07 13:28:30 -07:00
Harshit Bansal
25fa9a25ff emoji: Add support for animated GIF images.
This commit adds 'resize_gif()' function which extracts each frame,
resize it and coalesces them again to form the resized GIF while
preserving the duration of the GIF. I read some stackoverflow
answers all of which were referring to BiggleZX's script
(https://gist.github.com/BigglesZX/4016539) for working with animated
GIF. I modified the script to fit to our usecase and did some manual
testing but the function was failing for some specific GIFs and was not
preserving the duration of animation. So I went ahead and read about
GIF format itself as well as PIL's `GifImagePlugin` code and came up
with this simple function which gets the worked done in a much cleaner
way. I tested this function on a number of GIF images from giphy.com
and it resized all of them correctly.

Fixes: #9945.
2018-08-04 11:46:58 -07:00
Tim Abbott
31afa36d7b unminify: Clean up unnecessary repetition of webpack:/// URLs.
This takes stacktrace lines that used to look like this:

n@https://chat.zulip.org/static/webpack-bundles/app.2385793af60f0b082ee9.js:1:12680
       = webpack:///./static/js/blueslip.js line 241 column 1
dispatch@https://chat.zulip.org/static/webpack-bundles/app.2385793af60f0b082ee9.js:52:37878
       = webpack:////srv/zulip-npm-cache/8ea4cd291dd23441aec0f298b77b6ddc0d0a7a56/node_modules/jquery/dist/jquery.js line 5182 column 1

to have the even-numbered lines look like this:
       = ./static/js/blueslip.js line 241 column 1
dispatch@https://chat.zulip.org/static/webpack-bundles/app.2385793af60f0b082ee9.js:52:37878
       = /srv/zulip-npm-cache/8ea4cd291dd23441aec0f298b77b6ddc0d0a7a56/node_modules/jquery/dist/jquery.js line 5182 column 1
2018-08-03 15:56:40 -07:00
Yago González
936d8c32c0 test-api: Fix typo in message event types.
The event type for messages is "message", not "messages" as the test
(and hence the API docs Python example) reflected.
2018-08-02 15:54:41 -07:00
Yago González
2b8bb2ffc6 api docs: Migrate DELETE /events to OpenAPI. 2018-08-02 15:54:41 -07:00
Yago González
5631645d9f api docs: Migrate /register to OpenAPI. 2018-08-02 15:54:41 -07:00
Yago González
7d8e058e39 api docs: Implement an exception list for schema validation. 2018-08-02 15:54:41 -07:00
Tim Abbott
e7c7211c30 mypy: Fix type of messages in do_update_message_flags.
Ever since we moved the stream/everything cases to separate functions,
the messages argument has actually been required.
2018-08-01 17:37:16 -07:00
Tim Abbott
23246ff816 do_update_pointer: Switch to using where_unread().
This produces a more efficient database query (just because postgres
doesn't use the right index by default the other way).
2018-08-01 16:51:56 -07:00
Tim Abbott
c775be8ea4 do_mark_stream_messages_as_read: Accept a Client object.
We also fix an incorrect Optional in the type annotations.
2018-08-01 16:49:57 -07:00
Tim Abbott
6e55342e21 bulk_remove_subscriptions: Pass client object in.
We need the client object to pass on to do_mark_stream_as_read.
2018-08-01 16:48:31 -07:00
Tim Abbott
c60f197fde do_update_message_flags: Accept a Client object.
This is important for upcoming logging changes.
2018-08-01 16:40:58 -07:00
Tim Abbott
0e44010976 do_mark_all_as_read: Accept a client object.
This is needed for upcoming logging changes.
2018-08-01 16:40:15 -07:00
Tim Abbott
5f0519dfb4 do_update_pointer: Pass client object from callers.
We also fix an unused import.

This is needed for upcoming logging changes.
2018-08-01 16:40:15 -07:00
Tim Abbott
54d558b128 management: Add library for getting a client object.
This is to be used in some analytics features we're adding in the near
future.
2018-08-01 16:40:15 -07:00
Tim Abbott
e9f4d9db2b push_notifications: Fix interface for handle_remove_push_notification.
This really should just accept a message ID.
2018-08-01 16:36:42 -07:00
Kunal Gupta
bc43eefbfb notifications: Add function for cancelling GCM notifications.
This adds a new function called handle_remove_push_notification in
zerver/lib/push_notifications.py which requires user_profile id and
the message id which has to be removed in the function.

For now, the function only supports GCM (and is mostly there for
prototyping).

The payload which is being delivered needs to contain the narrow
information and the content of the message.
2018-08-01 15:59:04 -07:00
Tim Abbott
5f8d193bb7 notifications: Include realm_uri in push notifications.
This should make it much simpler for the mobile apps to line up the
data from server_settings against the data in the notifications.

Addresses part of #10094.
2018-08-01 15:46:15 -07:00
Tim Abbott
58ee3fa8c4 page_params: Include avatars and similar data in cross-realm bots.
This ensures that the format of this data structures matches that for
in-realm bots in the main users data structure (including avatars,
etc.).

Fixes #10138.
2018-08-01 15:09:11 -07:00
Tim Abbott
ccba1e7c0e cache: Limit which realms we access when filling caches.
For realms that don't have any presence-active users, we know for a
fact that there aren't any active clients that will be reloading just
after the server restarts, so we can skip filling the cache with data
related to that realm.

For zulipchat.com, this results in a significant performance
optimization for the recipient and stream caches, and a moderate
performance improvement for the user caches as well.
2018-08-01 14:22:49 -07:00
Tim Abbott
e6abc6e0bd cache: Only ensure we fill recipient caches for Recipient.STREAM.
Private messages make up the bulk of Recipient objects.  While private
messages are ~50% of messages, if you weight by messages received
(which is what is important for message-loading performance), it's
pretty strongly balanced towards stream messages.
2018-08-01 13:09:59 -07:00
Tim Abbott
463a348a86 cache: Don't pre-populate low-traffic user caches.
We only really need to pre-populate the caches used for (1) mobile
authentication and (2) most other user lookups.
2018-08-01 13:09:34 -07:00
Tim Abbott
c42302e47b restart-server: Optimize prepopulating user cache.
We don't need to include long-term idle or other inactive users here,
since fetching them consumed to vast majority of the time.

(On chat.zulip.org, this decreased the runtime for populating the user
cache by 5x, removing only users we're unlikely to need to access).
2018-08-01 12:54:06 -07:00
Tim Abbott
f10e006135 message: Bump MAX_UNREAD_MESSAGES to 50000.
This doesn't seem to have a huge performance downside (less than 1s
extra time for loading / on chat.zulip.org), and it means the
possibility of users having so many unreads that we get weird/buggy
behavior is much more unlikely to exist.

We'll still want a better experience for users who somehow go over
this limit, but it can be pretty firmly "you need to go mark some
things as read".
2018-08-01 12:02:54 -07:00
Rhea Parekh
ee37866687 import: Add gitter import file in zerver/data_import directory. 2018-08-01 11:52:14 -07:00
Rhea Parekh
b8e1e8b31d import: Add slack import files in zerver/data_import directory. 2018-08-01 11:52:14 -07:00
Vishnu Ks
6b3706494c notifications: Pass realm_creation argument to enqueue_welcome_emails. 2018-08-01 11:29:34 -07:00
Roman Godov
5e70577f84 models: Rename Realm.show_digest_email field.
This renames Realm.show_digest_email field to
digest_emails_enabled, for greater clarity as to what it does
just from seeing the setting name, without having to look it up.

Fixes part of #10042.
2018-08-01 11:05:58 -07:00
Tim Abbott
6f7e12ea19 docs: Add subsystem documentation for caching. 2018-07-31 17:00:45 -07:00
Tim Abbott
1b2a26ca83 events: Fix missing empty custom profile data dict for new users.
We were getting event-handling exceptions in JS in production if a new
user was created and then went and set a custom profile field, because
there was no `.profile_data` on their user object.  We were able to
trace the issue down to the fact that our events didn't include that
field when creating a new user.
2018-07-31 11:08:11 -07:00
Roman Godov
c0806917ec models: Rename Realm.restricted_to_domain field.
This renames Realm.restricted_to_domain field to
emails_restricted_to_domains, for greater clarity as to what it does
just from seeing the setting name, without having to look it up.

Fixes part of #10042.
2018-07-31 09:28:33 -07:00
Cynthia Lin
29442ffb93 zcommand: Add light/dark mode command aliases for day/night mode.
Fixes #10095.
2018-07-31 07:12:31 -04:00
Tim Abbott
e04156eef3 export: Fix error messages for stream list mismatches.
The previous error messages for this were written for a tool only to
be used by a couple people, and didn't make clear what potential
causes were.  Tweak these to provide greater clarity about what's
going on.

The main cause of these errors appearing in practice was fixed in
7ea5987e5d, but nothing strongly
prevents a similar issue from being introduced in the future.

Fixes #10078.
2018-07-30 22:32:26 -07:00
Tim Abbott
6317064210 unminify: Fix source map extraction for hash-named files.
Apparently, our old unminify logic relied on the fact that the
filenames displayed in tracebacks were of the form "app.js" (and the
`app.js` copy of the source map in the appropriate
`/home/zulip/deployments/`).  The correct behavior is to just look up
the source map for the appropriate hash-named
`app.a40806b10565c1dee5bf.js` type file.

We fix this with a few small tweaks to the regular expressions.  I
wish this file had reasonable unit tests.
2018-07-30 22:09:37 -07:00
Tim Abbott
aa5959396d docs: Add some basic subsystem documentation for thumbnailing. 2018-07-30 13:20:54 -07:00