Commit Graph

3618 Commits

Author SHA1 Message Date
Rhea Parekh
6a07897d3a slack importer: Fetch and save the avatars.
This gets the avatar of size 512 px and saves it in the
user's avatar directory with both the extensions
'.png' and '.original'.
2018-03-01 16:49:37 -08:00
Rhea Parekh
0f183981e6 Import script: Make sure medium avatars exist during import.
During a slack import, we don't have medium-size avatars already
available in the export data set (and possibly also with a normal
import/export?).  The medium size avatar can be created by the
'ensure_medium_avatar_image' function, which checks if the medium
image exists, and if it doesn't, it creates the image.

This commit was substantially edited by tabbott to get rid of an
undefined variable bug, avoid initializing the upload backend classes
in a loop, and add some TODO notes on things that could be improved
later.
2018-03-01 16:48:06 -08:00
Rhea Parekh
7076a49e04 slack importer: Pass avatar data through the import script. 2018-03-01 16:41:49 -08:00
Rhea Parekh
30b9d35d5e slack importer: Add helper functions to get user avatars.
Here, we create the slack avatar url using the user data and
build the avatar object. Added Tests for the same.
2018-03-01 16:38:55 -08:00
Rhea Parekh
95df8452be slack importer: Remove function 'get_user_avatar_source'.
slack avatar urls have the format:
'https://ca.slack-edge.com/<team_id>-<user_id>-<avatar_hash>-<size>'
For any url of this form, if the user hasn't uploaded an image,
Slack uses default gravatar, but we don't have a way of knowing if Slack
has used the uploaded image or the custom gravatar
eg: https://ca.slack-edge.com/T5YFFM2QY-U6006P1CN-gd41c3c33cbe-512.
Hence, avatar_source should be mapped to 'U'.
2018-03-01 16:38:55 -08:00
Greg Price
6d35a697af antispam: Don't let new accounts in open realms immediately send invites.
We haven't had a problem with this yet, but this should help prevent it.
2018-03-01 14:56:19 -08:00
rht
71ff8c370e django-2.0: Don't assign directly to Many-to-Many field.
The old pattern of setting the value and then using .save() here has
been deprecated.  set() also saves the record.
2018-03-01 08:49:35 -08:00
Robert Hönig
48b2e4eb66 backend: Add bot config data patching. 2018-03-01 08:25:43 -08:00
Robert Hönig
312cabb9a6 models.py: Move two long functions with circular imports to actions.py.
models.py should only contain thin wrapper functions. Furthermore,
this move allows us to remove the circular imports. The two moved
functions are interdependent and are thus moved in one commit.
2018-03-01 08:25:43 -08:00
Alena Volkova
822dfc6a34 integrations: Add webhook code, API endpoint, and tests for Front. 2018-02-28 16:18:19 -05:00
Vishnu Ks
120ac21515 bugdown: Create a constant for inline preview limit. 2018-02-28 12:53:36 -08:00
Aastha Gupta
d124597f4b org-settings: Add setting to turn off welcome emails.
This adds an organization-level setting to provide an option to turn
off the welcome emails.

Fixes #8000.
2018-02-28 12:39:01 -08:00
Robert Hönig
19bbc0e6f8 backend tests: Use correct bot owner in create_test_bot().
The correct owner is the one specified in the user_profile
parameter, which is different from self.user_profile.
2018-02-27 12:20:08 -05:00
Rhea Parekh
3bb14a867b slack importer: Change 'invite_only' mapping in streams.
'invite_only' should always be true for the slack's
standard export plan as the private channels are not
supported in it.
2018-02-25 09:22:01 -08:00
Rhea Parekh
a2f6f4ba1c slack importer: Handle case where messages have no users. 2018-02-25 09:20:55 -08:00
Rhea Parekh
15a6f62fe7 slack importer: Refactor defaultstream handling.
The check for the channel ('general' and 'random') must be added before
'build_defaultstream' function is called and then the id is incremented.
Otherwise, the id appended at the end of second defaultstream object, which would be
greater than the total number of defaultstream objects would crash at
'defaultstream_id_list[defaultstream_id]' which is a paramater of 'build_defaultstream'.
Added tests to prevent the same.
2018-02-25 09:20:55 -08:00
Eeshan Garg
0d39b05cf6 webhooks/beanstalk: Update text and remove screenshots.
This commit:

* Removes the unnecessary screenshot.
* Reorders the instructions and combines them in to 4 steps.
* Improves the contents of the webhook-url-with-bot-email-indented.md
  macro and makes it more consistent with create-bot-construct-url.md.
* Sets the recommended stream name to "commits", since that's what
  the webhook function for Beanstalk expects in
  zerver/webhooks/beanstalk/view.py. This allows us to use the
  create-stream.md macro.
2018-02-24 06:21:02 -05:00
Aditya Bansal
f50e325075 RealmAuditLog: Fill subscription events with event_last_message_id=None.
Creating the very first organization administrator user and
subscribing them to streams before any messages were sent resulted in
RealmAuditLog entries being created with a `event_last_message_id` of
None, because that's the maximum ID in the empty set.

We correct this by fixing the incorrectly created RealmAuditLog
entries, both for new servers and also fixing old broken entries on
existing servers.

This fixes an issue where if a user setup a Zulip server with just the
organization administrator, and then forgot about it (so that the
initial user became soft-deactivated), trying to sign in 3 weeks later
would throw an exception.

This fixes the issue reported here:

https://chat.zulip.org/#narrow/stream/9-issues/subject/500.20error.20on.20login/near/511981
2018-02-22 09:45:22 -08:00
Rhea Parekh
aff9099c3b slack importer: Get domain name from settings.EXTERNAL_HOST. 2018-02-21 08:58:27 -08:00
Rhea Parekh
f6d267e773 slack importer: Change 'total_users' to 'total_channels'.
The total number of stream objects are allocated to
total_users. They should be allocated to the total_channels.
This passed the tests as the total number of users in the test
where greater than the total number of channels.
2018-02-21 08:06:10 -08:00
Eeshan Garg
96036f07a3 api_code_example: Use json instead of ujson; specify separators.
The Markdown extension that lives inside
zerver/lib/bugdown/api_code_example.py previously used ujson.
ujson's `dumps` function doesn't accept a `separators` argument,
which means we have no control over how the JSON is pretty-printed.
This resulted in JSON fixtures with no spaces after the colon, which
looks unnecessarily convoluted.

So now, we use the built-in `json` module to get around this.

For further reading, this issue
<https://github.com/esnme/ultrajson/issues/82> opened on ujson's
repo explains why they are reluctant to support such formatting
due to performance considerations.
2018-02-20 20:10:38 -03:30
Eeshan Garg
e80609a3e9 api docs: Test sample fixture for invalid stream name error.
This commit adds a test for the sample fixture for when an invalid
stream name is passed to a query that expects a valid stream name
as an argument. This is the case with almost all of our queries
documented under the sidebar heading "Streams".

EDIT: Actually, I was wrong. This payload is highly specific to
get-stream-id, so it shouldn't be a part of common-error-payloads
at all.
2018-02-20 20:03:55 -03:30
Eeshan Garg
9add5bf6a7 api/delete-queue: Test fixture for BAD_EVENT_QUEUE_ID error.
In templates/zerver/api/delete-queue.md, we have a sample fixture
for when the queue_id passed to client.deregister_queue is not
valid or the event queue in question has already been deleted.
This commit tests that fixture.

Note that this error payload is specific to client.deregister_queue.
2018-02-20 14:54:12 -08:00
Eeshan Garg
a0818975fb api/create-user: Test fixture for "email already in use" error.
In templates/zerver/api/create-user.md, we have a sample fixture
for when a client attempts to create a user with the same email
as an existing user. This commit adds a test for that fixture.

Note that this error payload is specific to client.create_user
and this error payload isn't generated anywhere else.
2018-02-20 14:54:12 -08:00
Eeshan Garg
be0a04f33e api docs: Test sample fixtures for unauthorized_errors_fatal.
This commit adds tests for the sample fixtures for when
unauthorized_errors_fatal is passed to client.add_subscriptions.
2018-02-20 14:54:12 -08:00
Eeshan Garg
806476173f api/add-subscriptions: Test fixture for "already_subscribed".
In templates/zerver/api/add-subscriptions, we have a sample fixture
for when the user being subscribed is already subscribed to a
stream. This commit tests that fixture against a running server.
2018-02-20 14:54:12 -08:00
Eeshan Garg
06bf47d2f5 api docs: Test sample fixture for user not authorized error.
This commit adds tests for the fixture for when a user is not
authorized (perhaps because the query requires the use of admin
privileges) for a particular query.
2018-02-20 14:52:09 -08:00
Eeshan Garg
124c672a7c api docs: Test sample fixture for message edit permission error.
In templates/zerver/api/update-message.md, we have a sample fixture
for when a zulip.Client does not have the permission to update/edit
a particular message. This commit adds a test for that fixture.
Also, tools/test-api now also uses a non-admin client for this test,
which might come in handy in the future.
2018-02-20 14:52:09 -08:00
Eeshan Garg
1c32163bb9 api docs: Test sample fixture for missing request args error.
This commit adds tests for the sample fixture for when a required
request argument is missing. Also, it moves the sample fixture
to common-error-payloads.md, since this is an error payload that
is common to most requests (except the ones that don't take any
arguments).
2018-02-20 14:52:09 -08:00
Eeshan Garg
d1a5bbc2a5 api docs: Test sample fixture for invalid PM recipient error.
In templates/zerver/api/private-message.md, we have a sample fixture
for when the email address of the PM's recipient is invalid. This
commit makes sure that fixture is tested against a running server.
2018-02-20 14:52:09 -08:00
Eeshan Garg
93b44a41c5 api docs: Test sample fixture for nonexistent stream error.
In templates/zerver/api/stream-message.md, we have a sample fixture
for when the target stream does not exist. This commit adds a test
for that sample fixture.
2018-02-20 14:52:09 -08:00
nyan-salmon
03624b78c2 webhooks: Add Facebook integration. 2018-02-19 09:41:49 -08:00
neiljp (Neil Pilgrim)
4efb83ee29 mypy: Amend logic in check_upload_within_quota to avoid None. 2018-02-19 09:20:30 -08:00
Steve Howell
a474af8d58 Use newer stream urls in missed message emails.
This changes the missed-message logic to use the new encoding
scheme for stream url fragments that prefixes them with
"{stream_id}-".

For simplicity sake, we just get a Stream object to pass to
the helper functions, and we only call get_display_recipient()
now for the HUDDLE case.  (We were calling it wastefully before
for the PM case.)
2018-02-19 09:03:11 -08:00
Steve Howell
46a49777c4 Add stream ids to urls for stream-related narrows.
This commit prefixes stream names in urls with stream ids,
so that the urls don't break when we rename streams.

strean name: foo bar.com%
before: #narrow/stream/foo.20bar.2Ecom.25
after: #narrow/stream/20-foo-bar.2Ecom.25

For new realms, everything is simple under the new scheme, since
we just parse out the stream id every time to figure out where
to narrow.

For old realms, any old URLs will still work under the new scheme,
assuming the stream hasn't been renamed (and of course old urls
wouldn't have survived stream renaming in the first place).  The one
exception is the hopefully rare case of a stream name starting with
something like "99-" and colliding with another stream whose id is 99.

The way that we enocde the stream name portion of the URL is kind
of unimportant now, since we really only look at the stream id, but
we still want a safe encoding of the name that is mostly human
readable, so we now convert spaces to dashes in the stream name.  Also,
we try to ensure more code on both sides (frontend and backend) calls
common functions to do the encoding.

Fixes #4713
2018-02-19 09:03:11 -08:00
Rhea Parekh
b702bbe5a1 slack importer: Allocate ids in a single db query.
We use the command
'select nextval('sequence') from generate_series(1, increment_number)'
which returns a list of allocated values for the ids.

This list is used to assign ids to the to be converted objects.
2018-02-19 08:55:50 -08:00
Rhea Parekh
5dfacfcfca slack importer: Change 'allocate_ids' to return a list of ids.
Update the callers of this function to process the list and add
tests for the same.
2018-02-18 20:47:45 -08:00
Shubham Dhama
9feae472f8 org settings: Add button to deactivate organization.
This adds button under "Organization profile" settings, which
deactivates the organization and sends an "event" to all the
active user and log out them.

Fixes: #8212.
2018-02-18 10:20:38 -08:00
Greg Price
693a9a5e70 push notifs: For group PMs, identify the users in the group. 2018-02-16 16:06:03 -08:00
Shubham Dhama
a32e1eb913 markdown: Require double-asterisk around all mentions.
This enforces `**` around all the mentions including "at-all" and
"at-everyone" mentions. Hence this makes `@all` and `@everyone`
invalid mentions, resulting into proper syntax for these mentions as
`@**all**` and `@**everyone**` respectively.

Note from tabbott: This removes an old feature/syntax, which made
sense back when @Tim was also a way to mention a user with Tim as
their first name.  Given how nice typeahead is now, the user part of
the feature was removed a while ago; this should have gone at the same
time.

Fixes: #8143.
2018-02-16 11:45:08 -08:00
Eeshan Garg
6206647641 bugdown/api_code_examples: Add support for multiple languages.
This commit modifies the Markdown extension in bugdown/api_code_examples.py
to support rendering code examples in multiple languages by specifying
the language like so:

{generate_code_example(python)|doc.md|example}

This makes us one step closer towards adding support for testable
JavaScript code examples.
2018-02-16 10:07:13 -08:00
Eeshan Garg
73a1f8aa3e api docs: Add comment about why the event queue example isn't tested.
This commit simply adds a comment in zerver/lib/api_test_helpers.py
explaining why it isn't worth the effort to explicitly test the
code example in api/get-events-from-queue.md.
2018-02-16 10:07:13 -08:00
Shubham Padia
a4b686297a api: Return anchor in get_messages when use_first_unread_anchor=True.
This may be helpful for some API clients, since it avoids them needed
to do somewhat messy post-processing on the results (the data was
always available via scanning for the first unread message in the result).

Fixes #6244.
2018-02-16 10:06:20 -08:00
Robert Hönig
a19a69bfe3 embedded bots: Log warning when bot quit()s.
External bots may call bot_handler.quit() when
they wish to terminate, e.g. due to a misconfiguration.
Currently, embedded bots ignore calls to quit(), even
though they signal a problem. This commit does the first
step in handling quit() calls by logging a warning.
2018-02-13 14:56:37 -08:00
neiljp (Neil Pilgrim)
756af7518c REQ: Add type parameter to act as hint for typing. 2018-02-13 11:40:52 -08:00
neiljp (Neil Pilgrim)
ec829ea832 mypy: Add 2 Optional[StreamTopicTarget] annotations in actions.py. 2018-02-13 11:40:52 -08:00
neiljp (Neil Pilgrim)
79582a59f1 mypy: Move assertion into similar branch in get_recipient_info. 2018-02-13 11:40:52 -08:00
neiljp (Neil Pilgrim)
3cb12230b2 mypy: Annotate email_gateway_bot in export_files_from_s3(). 2018-02-13 11:40:52 -08:00
neiljp (Neil Pilgrim)
16b939a964 REQ: Ensure only initial parameter is positional & adjust views/messages.py. 2018-02-13 11:40:52 -08:00
neiljp (Neil Pilgrim)
b0126ca8dc mypy: Amend diff_strings to return str or raise DiffException. 2018-02-13 11:40:51 -08:00