Commit Graph

3762 Commits

Author SHA1 Message Date
Vishnu Ks
57e0a24020 management: Scrub delivery email as well while running scrub_realm. 2018-10-16 11:16:46 -07:00
Kevin Lin
b2c29274f3 presence: Allow bots to fetch realm presence data.
Before, presence information for an entire realm could only be queried via
the `POST /api/v1/users/me/presence` endpoint. However, this endpoint also
updates the presence information for the user making the request. Therefore,
bot users are not allowed to access this endpoint because they don't have
any presence data.

This commit adds a new endpoint `GET /api/v1/realm/presence` that just
returns the presence information for the realm of the caller.

Fixes #10651.
2018-10-15 12:51:28 -07:00
Steve Howell
76deb30312 preview: Hash cache keys for preview urls.
We don't want really long urls to lead to truncated
keys, or we could theoretically have two different
urls get mixed up previews.

Also, this suppresses warnings about exceeding the
250 char limit.

Finally, this gives the key a proper prefix.
2018-10-14 09:28:57 -07:00
Tim Abbott
68ab71eb8b push: Fix exceptions when removing push notifications.
Now that we allow multiple users to have registered the same token, we
need to configure calls to unregister tokens to only query the
targeted user_id.

We conveniently were already passing the `user_id` into the push
notification bouncer for the remove API, so no migration for older
Zulip servers is required.
2018-10-12 11:19:23 -07:00
Steve Howell
0f7628280f narrow: Handle spurious emails in pm-with searches.
If cordelia searches on pm-with:iago@zulip.com,cordelia@zulip.com,
we now properly treat that the same way as pm-with:iago@zulip.com.

Before this fix, the query would initially go through the
huddle code path.  The symptom wasn't completely obvious, as
eventually a deeper function would return a recipient id
corresponding to a single PM with @iago@zulip.com, but we would
only get messages where iago was the recipient, and not any
messages where he was the sender to cordelia.

I put the helper function for this in zerver/lib/addressee, which
is somewhat speculative.  Eventually, we'll want pm-with queries
to allow for user ids, and I imagine there will be some shared
logic with other Addressee code in terms of how we handle these
strings.  The way we deal with lists of emails/users for various
endpoints is kind of haphazard in the current code, although
granted it's mostly just repeating the same simple patterns.  It
would be nice for some of this code to converge a bit.  This
affects new messages, typing indicators, search filters, etc.,
and some endpoints have strange legacy stuff like supporting
JSON-encoded lists, so it's not trivial to clean this up.

Tweaked by tabbott to add some additional tests.
2018-10-12 10:18:30 -07:00
Steve Howell
51bd36e448 tests: Add coverage to get_service_interface_class(). 2018-10-11 16:12:07 -07:00
Steve Howell
8379aeee15 outgoing bots: Fix header for generic servers.
For our bots that use GenericOutgoingWebhookService
(which are basically Zulip style bots), we now
include a "content-type" header of "application/json".

We accomplish this by having the service classes
implement their own custom method called
`send_data_to_server`. For the Slack-related
code, we just extracted code from `do_rest_call`,
and then for the Zulip-related code, we added
a `headers` parameter.
2018-10-11 16:12:07 -07:00
Steve Howell
8226e13e9c bot tests: Replace use of MockServiceHandler.
We'll just use a real class here, since the service
handlers are pretty lightweight and just munge data.
2018-10-11 16:12:07 -07:00
Steve Howell
31597cf33e Remove timeout parameter in do_rest_call().
Nobody was setting it.
2018-10-11 16:12:07 -07:00
Steve Howell
69ee84bb14 refactor: Extract build_bot_request().
This fixes a couple things:

    * process_event() is a pretty vague name
    * returning tuples should generally be avoided
    * we were producing the same REST parameters in both
      subclasses
    * relative_url_path was always blank
    * request_kwargs was always empty

Now process_event() is called build_bot_request(),
and it only returns request data,
not a tuple of `rest_operation` and `request_data`.

By no longer returning `rest_operation`, there are
fewer moving parts.  We just have `do_rest_call` make
a POST call.
2018-10-11 16:12:07 -07:00
Steve Howell
16eff75e49 refactor: Simplify how we use base_url.
Before this change, we instantiated base_url into a superclass
of subclasses that returned base_url into a dictionary that
gets returned to our caller.

Now we just pull base_url out of service when we need to make
the REST call.
2018-10-11 16:12:07 -07:00
Steve Howell
b89a94f730 Improve errors when we can't connect to a bot server.
We don't overwhelm people with error info when bots
fail to connect or time out.
2018-10-11 16:12:07 -07:00
Steve Howell
3790c469e9 outgoing bots: Report JSON errors to users.
We should arguably report these to bot owners
as well, but this is at least an improvement
over having the server crash.
2018-10-11 16:12:07 -07:00
Steve Howell
df4b665658 refactor: Parse JSON from bots in one place.
We move the JSON parsing step into the
higher level function: process_success_response().

In the unlikely event that we'll start integrating
with a solution that doesn't use JSON, we can deal
with that, and for now doing the parsing in one
place will help us make error reporting more
consistent.

In a subsequent commit we'll introduce better
error handling for malformed JSON.
2018-10-11 16:12:07 -07:00
Steve Howell
229dd5d861 outgoing webhooks: Get rid of "Success!" prefix.
The earlier code here, if it got a payload with
"response_string" as a key, would prefix the
corresponding value with "Success!".  We just
want the bot to set its own content.

The code is reorganized here so that process_success()
always produces a value keyed by "content" from
incoming data, and then process_success_response()
doesn't do any fancy munging of the data.
2018-10-11 16:12:07 -07:00
Tim Abbott
8cf104b643 avatar: Allow API authentication for /avatar/ routes.
This makes it feasibly for the mobile apps to correctly render user
avatars generated by the `!avatar()` syntax.
2018-10-11 15:52:29 -07:00
Aditya Bansal
6893f52ad9 thumbnails: Instruct thumbor to sharpen thumbnailed images.
Fixes: #10218.
2018-10-11 15:44:47 -07:00
Aditya Bansal
6e433186a1 thumbnails: Change thumbnail size to be 300px.
Fixes: #10219.
2018-10-11 15:44:47 -07:00
Vishnu Ks
962d72b58b retention: move_messages_to_archive should accept multiple message ids.
This will speed up the scrub realm management command. Calling the
function with a single message_id in a loop was extremely inefficient.
2018-10-11 15:31:12 -07:00
Vishnu Ks
6972de21be management: Add command to scrub a realm of personal data. 2018-10-11 15:30:26 -07:00
Vishnu Ks
5bdadc8061 upload: Create function to delete avatar image. 2018-10-11 15:30:26 -07:00
Vishnu Ks
6aa4b64dc0 emails: Don't log emails while running test suite.
Modified the tests to ensure 100% coverage.
2018-10-11 15:12:08 -07:00
Vishnu Ks
d8c19cb003 models: Move billing models from zilencer to corporate. 2018-10-11 14:54:29 -07:00
Steve Howell
4956107c53 refactor: Simplify return type for process_success().
There's no reason to return a failure message in
process_success(), since it's implied to be part of
the success codepath.  I didn't look at the full history
of how the strange API evolved, but the second element
of the tuple was clearly noise by the time I got here.
Neither of the subclasses ever set it, and none of the
consumers used it.
2018-10-09 15:56:24 -07:00
Steve Howell
f2dd218331 refactor: Inline succeed_with_message().
This two-line function wasn't really carrying its
weight, and it just made it harder to refactor the
overall codepath.

Eliminating the function forces us to mock at a slightly
deeper level, which is probably a good thing for what
the test intends to do.  The deeper mock still verifies that
we're sending the message (good) without digging into
all the details of how we send it (good).

Note that we will still keep around the similarly named
`fail_with_message` helper, which is a lot more useful.
(The succeed/fail scenarios aren't really symmetric here.
For success, there are fewer codepaths that do more complex
things, whereas we have lots and lots of failure codepaths
that all do the same simple thing of replying with a canned
message.)
2018-10-09 15:56:24 -07:00
Steve Howell
fa505a1af1 refactor: Have process_success return structured data.
Before this change subclasses of OutgoingWebhookServiceInterface
would return a raw string as the first element of its return
tuple in process_success().  This is not a very flexible
design, as it prevents the bot from passing extra data like
`widget_content`.

It's also possible in the future that we'll want to let outgoing
bots reply directly to senders who mention them on streams, and
again the original design was overly constrained for that.

This commit does not actually change any functionality yet.
2018-10-09 15:56:24 -07:00
Callum Fraser
383f1633e6 api: Add limit of total messages available per request of 5000.
Tweaked by tabbott to use a declared constant rather than just use
5000 in multiple places; this also means we can change the count
without updating translations.

Fixes #10446.
2018-10-09 15:43:21 -07:00
Vishnu Ks
d8ddb7f69b settings: Remove reminder-bot from REALM_INTERNAL_BOTS. 2018-10-02 11:12:07 -07:00
Rishi Gupta
7f16d27015 api docs: Update api-keys. 2018-09-25 21:02:58 -07:00
Harshit Bansal
20aee7b116 emoji: Change fallback emojiset for text emojiset.
Now that we have added back support for `google-blob` emojiset,
changing the fallback emojiset for `text` emojiset to `google-blob`
make sense.
2018-09-21 18:34:12 -07:00
Shubham Padia
6bfa29b8e6 notifications: Fix soft-deactivated users don't get push notifications.
Fixes the urgent part of #10397.

It was discovered that soft-deactivated users don't get mobile push
notifications for messages on private streams that they have configured
to send push notifications.

Reason: `handle_push_notification` calls `access_message`, and that
logic assumes that a user who is a recipient of a message has an
associated UserMessage row. Those UserMessage rows are created
lazily for soft-deactivated users, so they might not exist (yet)
until the user comes back.

Solution: Ensure that userMessage row is created for
stream_push_user_ids and stream_email_user_ids in create_user_messages.
2018-09-21 12:06:18 -07:00
Rishi Gupta
b470cef864 import: Set Realm.plan_type to SELF_HOSTED on import.
Tweaked by tabbott to avoid an unnecessary .save().
2018-09-21 10:57:22 -07:00
Robert Spier
f4c82a1153 emails: Add detection for ChromeOS User-Agents. 2018-09-21 10:41:05 -07:00
Tim Abbott
feee76eb23 export: Fix exporting files with S3 upload backend.
At some point as part of the process of supporting renumbering data,
we changed the structure of our file uploads to expect `path` to match
`s3_path`, with both having the relative path within the overall
hierarchy (including the realm ID).  This change updates the more
rarely-used S3 export code path to use that model, fixing a crash when
messages reference an Attachment object with a rewritten path_id.
2018-09-20 20:14:19 -07:00
Eeshan Garg
4f366daec0 markdown: Add extension for creating tabbed sections on /help and /api. 2018-09-18 13:49:34 -07:00
Rishi Gupta
44ea5f8492 user docs: Update message-a-stream-by-email.
Note we're no longer using subscriptions_html in the help docs, so no need
to test for it. There is already a test for subscriptions_html in
IntegrationTest.
2018-09-15 23:57:22 -07:00
Rishi Gupta
a87ed67695 user docs: Add relative links for gear menu items. 2018-09-15 23:47:45 -07:00
Rishi Gupta
7acb4e9165 portico: Update text for reset_emailed.html.
The current text may have made sense with different styling, but it feels
harsh now.
2018-09-15 18:45:10 -07:00
Eeshan Garg
7a755ff520 models: Add get_user_by_id_in_realm_including_cross_realm().
Ongoing efforts surrounding #9474 and #10360 are contingent upon
such a function being available.
2018-09-12 11:43:24 -07:00
Vishnu Ks
169de2f9cb billing: Include link to /plans and /billing in gear menu. 2018-09-07 17:30:26 -07:00
Rishi Gupta
2a0523cbe5 billing: Hide /plans if Realm.plan_type=SELF_HOSTED. 2018-09-05 13:44:35 -07:00
Vishnu Ks
c913eafdf4 portico: Change buttons on /plans to reflect current plan. 2018-09-05 13:44:35 -07:00
Vishnu Ks
9489ce0efc plans: Redirect to login if /plans is accessed from realm subdomain. 2018-09-05 13:44:35 -07:00
Rohitt Vashishtha
2864ce552b bugdown: Add @user|id syntax to support users with same name. 2018-08-31 14:16:47 -07:00
Rohitt Vashishtha
bc37800ad5 bugdown: Store users with same full_names in mention_data.
We start by stripping the ids in front of the name before the database
lookup. This has the advantage of not mentioning anyone if an incorrect
user id and full name combination is specified, as well as not having
the query the database twice, once by fullname and next by id.

Previously, we were storing only the most recent person with the same
full name as others; this commit adds new keys to the dict such that
simply looking by name would get you the newest user with this name,
and the get_user_by_id function can index the remaining users.
2018-08-31 14:16:47 -07:00
Harshit Bansal
e7311cdf5d emoji: Bring back the feature of changing emojisets.
This is largely inspired by requests from people not liking the
Google's new emojiset. A lot of people were requesting to revert
back to old blobs emojiset so we are re-enabling this feature
after making relevant infrastructure changes for supporting google's
old blob emojiset and re-adding support for twitter emojiset.

Fixes: #10158.
2018-08-31 11:52:24 -07:00
Shubham Padia
69bfa8c432 auth: Use different defaults for name and email for fakeldap.
Fixes part of #10297.
Use FAKE_LDAP_NUM_USERS which specifies the number of LDAP users
instead of FAKE_LDAP_EXTRA_USERS which specified the number of
extra users.
2018-08-31 17:07:02 +05:30
Pragati Agrawal
ba4e4e38ae user settings: Add setting to make login notifications optional.
This adds a feature in the "Notification" section of "Settings" tab,
which lets user enable or disable login emails notification.

Tweaked by tabbott to simplify the test.

Fixes: #5795, progress towards #5854.
2018-08-28 14:02:31 -07:00
Tim Abbott
54e90deda8 notifications: Handle APNS "Unregistered" errors properly.
Apparently, the APNS library we're using is inconsistent about the
format of its result entries; some are strings while others are
tuples.
2018-08-28 12:12:46 -07:00
Roman Godov
a2f407c11b api: Add an endpoint for listing the user groups in realm.
Adds endpoint for listing the user groups in realm.
Provides "description", "members", "name" and "id" for each user group.

Fixes #10298
2018-08-27 16:56:23 -07:00