Commit Graph

3618 Commits

Author SHA1 Message Date
Aditya Bansal
77651ece39 thumbnails: Rename size value 'original' to 'full'. 2018-07-30 13:00:23 -07:00
Aditya Bansal
5b5d8bb310 thumbnails: Rename data-original to data-src-fullsize. 2018-07-30 13:00:23 -07:00
Tim Abbott
02ae71f27f api: Stop using API keys for Django->Tornado authentication.
As part of our effort to change the data model away from each user
having a single API key, we're eliminating the couple requests that
were made from Django to Tornado (as part of a /register or home
request) where we used the user's API key grabbed from the database
for authentication.

Instead, we use the (already existing) internal_notify_view
authentication mechanism, which uses the SHARED_SECRET setting for
security, for these requests, and just fetch the user object using
get_user_profile_by_id directly.

Tweaked by Yago to include the new /api/v1/events/internal endpoint in
the exempt_patterns list in test_helpers, since it's an endpoint we call
through Tornado. Also added a couple missing return type annotations.
2018-07-30 12:28:31 -07:00
Shubham Dhama
499a70b01e notifications: Fix leaking of private stream msgs sent before subscribing.
Fixes: #9834.
2018-07-28 15:13:08 -07:00
Shubham Dhama
c5d8fdf68c message: Add function to check message access in bulk.
This effectively just calls access_message in a loop.
2018-07-28 15:12:55 -07:00
Tim Abbott
6bbffe0e2e notifications: Extract zerver/lib/url_encoding.py.
Extracting this helper library will help us avoid an import loop
between notifications.py and message.py (with bugdown in between).

But in addition to that, it's a more natural model, since some of the
uses for these functions weren't part of the notifications code
anyway.
2018-07-28 15:12:55 -07:00
Shubham Dhama
605a6ed3d8 message: Extract function for message access by a user. 2018-07-28 14:58:36 -07:00
Shubham Dhama
9cf67a5e71 models: Add function to get UserMessage from message id. 2018-07-28 14:58:36 -07:00
Tim Abbott
38dd9e49de import_realm: Add comments for update_model_ids.
This basically explains why in these cases we delay doing
bulk_import_model.
2018-07-26 16:13:14 -07:00
Rhea Parekh
fca6bc91aa import: Add 'get_db_table' function.
Implement this function in 'bulk_import_model'
and 'update_model_ids'.

This lets us save on redundant-feeling arguments in these
frequently-called helper functions.
2018-07-26 16:07:31 -07:00
Roman Godov
064fca5b76 import: Fix use of unused Subscription.notifications.
This code should have been interacting with the specific sub-fields
for notifications.
2018-07-26 15:53:29 -07:00
Yago González
db81726bcb api docs: Use method from our Python bindings in the example.
One of the code examples for GET /users was using the raw call_endpoint
method from our Python bindings, rather than get_members, which has been
specifically designed to interact with this endpoint.

Now all the examples here use the appropiate method.
2018-07-26 15:34:31 -07:00
Yago González
40a21f6077 api docs: Migrate GET /users to OpenAPI. 2018-07-26 15:34:31 -07:00
Yago González
7c50f6cdd4 api docs: Migrate POST /messages/render to OpenAPI. 2018-07-26 15:34:31 -07:00
Yago González
57c2d8c72a api docs: Migrate GET /get_stream_id to OpenAPI. 2018-07-26 15:34:31 -07:00
Yago González
679319a735 api docs: Migrate GET /streams to OpenAPI. 2018-07-26 15:34:31 -07:00
Yago González
95ce311686 api docs: Display the default value of parameters when present.
Whenever a parameter for an endpoint in our REST API has a default
value, it is displayed under the "Description" section of the
arguments table in the docs.

This way, we don't need to explicitly indicate the default values in the
description, thus avoiding duplicate information in the OpenAPI source.
2018-07-26 15:34:31 -07:00
Yago González
7ddc6e8d10 api docs: Migrate POST /messages to OpenAPI. 2018-07-26 10:24:01 -07:00
Yago González
cb24756edc api docs: Merge docs for sending stream and private messages. 2018-07-26 10:24:01 -07:00
Shubham Dhama
50ed378dd6 bugdown: Improve exception handler for BugdownRenderingException.
This has the benefit that we now get the usual data about the
user/request/etc. in error emails related to bugdown exceptions;
previously we were just getting the traceback in the emails (since our
`mail_admins` template was very simple) and no other debugging
details.

Comments tweaked by tabbott to help make clear exactly what's going on
here, since it's a little subtle and a little hacky.

Fixes #8843.
2018-07-26 09:27:55 -07:00
Shubham Dhama
a7e913894c bugdown: Add bugdown_logger for logging. 2018-07-26 09:27:55 -07:00
Tim Abbott
5671ea16ab bugdown: Fix buggy rendering of pre elements inside bullets.
Our nested code block processor wasn't using the correct test for
whether a paragraph was empty of other content; first, we need to
confirm no children, and second, we need to confirm there is no text
before/after the code block element inside the p tag.
2018-07-24 12:34:36 -07:00
Tim Abbott
3b36532407 storage: Fix hash-renaming of various non-CSS/JS file types.
None of the file types here are actually processed by our static asset
pipeline in a way that would result in the hash-named versions of the
files (stored in staticfiles.json) being accessed.  (If they were,
we'd be using something like `render_bundle` to access their paths).

So, we should not be generating/shipping these hash-named versions of
image, audio, and locale files.

This change decreases the size of a Zulip release tarball from 153MB
to 93MB, by removing all of the duplicated copies of various asset
files.

This may also help with #10038, but I'm not marking it as completing
that issue yet, because part of #10038 is that the non-hash-named
image files in prod-static/generated/emoji do not seem to have been
properly overwritten on upgrade, and it's unclear why.

Fixes #5971.
2018-07-24 09:42:25 -07:00
Rishi Gupta
e37b5dce79 stream settings: Fix brand new streams showing 0 weekly traffic.
A stream created in the last few hours likely won't be in StreamCount
(since that gets updated once a day), and hence won't be in the
recent_traffic dict.

However, get_average_weekly_stream_traffic should be None in this case,
not 0.
2018-07-23 16:43:15 -07:00
Rishi Gupta
0573c17819 stream settings: Set weekly traffic to None instead of -1 for new streams.
The only place this gets used seems to be subscription.handlebars, where a
value of None (or null) is probably less broken-seeming than -1 anyway.
2018-07-23 16:43:15 -07:00
Rishi Gupta
b75aa8e35a stream settings: Fix age calculation in is_old_stream.
Match the calculation in get_average_weekly_stream_traffic.
Regression introduced in dc7cfd3.

Fixes #10002
2018-07-23 16:43:15 -07:00
Joshua Pan
a9d9919648 Add delivery_email field to page_params.
This allows the frontend to use the `delivery_email`
field for display use.
2018-07-23 15:38:41 -07:00
Harshit Bansal
c0b0fb7cce emoji: Move EMOTICON_CONVERSIONS mapping to build_emoji infra.
This commit closes a long pending issue which involved moving the
`EMOTICON_CONVERSION` mapping to build_emoji infrastructure so
that there is only one source of truth. This was pending from the
time when this feature was implemented.
2018-07-23 12:35:08 -07:00
Rhea Parekh
8803ac4af8 import: Fix BotStorageData and BotConfigData import.
The function 'update_model_ids' should be used on
the models BotStorageData and BotConfigData.
It is wrongly added here for UserGroup model.

Also the sequence name for BotStorageData and
BotConfigData is 'zerver_botuserstatedata_id_seq' and
'zerver_botuserconfigdata_id_seq' respectively, which
should be specifically mentioned in the function
'allocate_ids'.

This fixes some nondeterministic test failures.
2018-07-23 11:24:17 -07:00
Tim Abbott
839300d781 import: Fix typo in re_map_foreign_keys_many_to_many.
This was introduced when I rebased together the two implementations of
this function.
2018-07-23 10:19:04 -07:00
Rhea Parekh
a656257e6b gitter import: Process user mentions.
The gitter mentions are in the format '@usermention'
and the mentions are included in the export data as:

"mentions": [
   {
    "screenName": "usermention",
    "userId": "54d7876c15522ed4b3dbbefb",
    "userIds": []
   }]

We extract this data and map this mention to @**usermention**
for Zulip.
2018-07-23 08:36:30 -07:00
Rhea Parekh
de13704d0b gitter import: Write messages batch-wise.
Messages can be bulky, and storing them in a single
data structure can cause a memory error.

In this commit, the messages are written to a file
batch-wise, thus avoiding the memory error.

Similar to commit 6b7b6b38ad
2018-07-23 08:36:30 -07:00
Rhea Parekh
3a6921e54b gitter import: Add gitter data conversion script. 2018-07-23 08:36:30 -07:00
Tim Abbott
db1260fb93 export: Clean up comments on why tables are not currently exported. 2018-07-23 08:28:20 -07:00
Rhea Parekh
c42d6b6983 export: Remove 'zerver_pushdevicetoken' from the to be imported list.
PushDevicetoken is automatically created when a user logs
in a server from mobile. This shouldn't be imported.
2018-07-23 08:21:00 -07:00
Rhea Parekh
fe299277f3 import: Import BotStorageData and BotConfigData. 2018-07-23 08:21:00 -07:00
Rhea Parekh
f01ff28e03 export: Export BotStorageData and BotConfigData. 2018-07-23 08:21:00 -07:00
Rhea Parekh
2978e025df import: Import UserGroup. 2018-07-23 08:21:00 -07:00
Rhea Parekh
98a7762a51 export: Export user groups. 2018-07-23 08:20:58 -07:00
Rhea Parekh
bc2307108d import: Import Service. 2018-07-23 08:20:58 -07:00
Rhea Parekh
0fcf6d9a40 export: Export Service. 2018-07-23 08:20:58 -07:00
Rhea Parekh
f444e5b628 import: Import MutedTopic. 2018-07-23 08:20:58 -07:00
Rhea Parekh
6eab6446fc export: Export MutedTopic. 2018-07-23 08:20:58 -07:00
Rhea Parekh
091d101e7d import: Import UserHotspot. 2018-07-23 08:20:58 -07:00
Rhea Parekh
8897e187c0 export: Export UserHotspot. 2018-07-23 08:20:58 -07:00
Rhea Parekh
6f7b7e143f import: Map user IDs for custom profile field of type 'USER'.
The CustomProfileField object which has the `field_type`
`USER` needs to be updated with the new user IDs.
2018-07-23 08:19:04 -07:00
Rhea Parekh
3879e345d9 import: Add function to re-map foreign keys for ManyToMany fields.
This will be used while for any ManyToMany field which
is being imported.
We add an internal function which takes in the old ID list
of the ManyToMany field and return the new updated ID list.
2018-07-23 08:19:04 -07:00
Tim Abbott
bdc95b5d72 slack webhooks: Eliminate unnecessary use of Service model.
The user ID of the bot user is a much more useful value to be sending
to the third-party API here.
2018-07-23 08:12:08 -07:00
Vishnu Ks
2b28042ddf models: Rename remaining audit log event types to past tense.
This makes the realm audit event type log entries more consistent.
2018-07-22 20:00:28 -07:00
Anders Kaseorg
a0293e8a86 zerver/lib/import_realm.py: Avoid shelling out for mkdir.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-19 10:43:37 -07:00