Commit Graph

92 Commits

Author SHA1 Message Date
Tim Abbott
a2f49b425b slack import: Fix some erroneous UserProfile field values.
* tutorial_status was the invalid value 'T'; should be the default of 'W'.
* last_reminder can be just the default None
* enable_desktop_notifications was just the model default of True.
2018-05-01 09:14:47 -07:00
Rhea Parekh
f00b80058d slack import: Remove unwanted comments. 2018-05-01 09:09:36 -07:00
Rhea Parekh
e579bef8fd slack import: Improve how we construct user_profile objects.
Fixes #9260
2018-05-01 09:09:36 -07:00
Tim Abbott
7d6bb3dcb4 settings: Remove obsolete default_desktop_notifications setting.
This actually hasn't been hooked up to do anything in years.

While we're at it, we remove the entire "Zulip Labs" settings page.
2018-04-28 13:46:07 -07:00
Tim Abbott
ff9371d63c slack import: Fix issues with Slack empty files.
Fixes #9217.
2018-04-25 10:20:55 -07:00
Preston Hansen
efc7967355 slack import: Update build_zerver_realm to use Realm defaults.
Fixes #9131.
2018-04-23 17:15:51 -07:00
Tim Abbott
7dfa0edfa6 slack import: Don't try to import pinned/unpinned items.
There isn't a corresponding Zulip concept, and they don't have a
"text" attribute, so there's no message content to import.
2018-04-18 11:36:31 -07:00
Tim Abbott
acd3a364e1 slack import: Refactor handling of dropped messages.
This is a more coherent ordering, because some messages we skip lack a
"text" attribute.
2018-04-18 11:35:59 -07:00
Tim Abbott
e759fd9be4 slack import: Improve error handling for invalid messages. 2018-04-18 11:29:40 -07:00
Tim Abbott
1410a1e460 slack import: Remove unnecessary zerver_realm_skeleton.json.
This was stored as a fixture file under zerver/fixtures, which caused
problems, since we don't show that directory under production (as its
part of the test system).

The simplest emergency fix here would be to just move the file, but
when looking at it, it's clear that we don't need or want a fixture
file here; we want a Python object, so we just do that.

A valuable follow-up improvement to this block would be to create an
actual new Realm object (not saved to the database), and dump it the
same code we use in the export tool; that should handle the vast
majority of these correctly.

Fixes #9123.
2018-04-18 10:33:53 -07:00
Rhea Parekh
a2070fb7e5 slack importer: Add comment on size information of avatars.
The size information of an avatar is not required during the import.
Check function 'import_uploads_local' and 'import_uploads_s3'
in 'export.py' for this.
2018-04-16 14:44:57 -07:00
Rhea Parekh
f6b6aa1e75 slack import: Implement threading as a management command. 2018-04-15 19:53:02 +05:30
Rhea Parekh
7c0c3930a8 slack importer: Thread avatar downloads. 2018-04-15 19:53:01 +05:30
Rhea Parekh
ebc2ee28e9 slack importer: Thread emoji downloads. 2018-04-15 19:52:59 +05:30
Rhea Parekh
8a291d0232 slack importer: Thread attachment downloads.
Use Zulip's run_parallel method to run thread downloads.
2018-04-15 19:51:58 +05:30
Rhea Parekh
f7398cbb09 slack import: Implement custom profile fields.
Add custom profile fields in the slack converted
data 'realm' file.
Added tests for the custom profile fields.

Fixes #8928
2018-04-10 13:28:53 -07:00
Rhea Parekh
852e8516b4 slack import: Add custom profile fields.
Build CustomProfileField and CustomProfileFieldValue
for every user and process the field type after getting an
entire list of the custom fields.
2018-04-10 13:28:53 -07:00
rht
a183186672 slack importer: User session.get to recycle previous connections. 2018-04-09 22:02:01 -07:00
rht
630adb406b Slack importer: Map Slack channel mentions to Zulip stream mentions. 2018-04-09 10:47:39 -07:00
Tim Abbott
c41d7ee300 slack import: Write reasonable multi-line JSON.
This is a lot better for debugging.
2018-04-09 10:45:35 -07:00
Rhea Parekh
2baa9bc16e Import: Add subdomain in the import script.
Also remove user input of subdomain in the slack data
conversion script.
2018-04-06 09:12:56 -07:00
Rhea Parekh
f4ad464d82 import script: Fix broken links to attachments.
The comments explain this pretty well, but basically because we
rewrite the realm ID during the import process, we need to edit all
the message bodies that link to an attachment to instead link to the
post-processed URL where that file will be hosted on the new server.

Fixes #8926.
2018-04-04 10:05:15 -07:00
Rhea Parekh
5a9cea4134 import script: re map foreign key of UserProfile.last_active_message_id. 2018-04-04 08:53:09 -07:00
Rhea Parekh
1bba6cc4ce slack importer: Support custom emoji reactions. 2018-04-01 23:24:35 -07:00
Rhea Parekh
2c116e46ce slack importer: Get custom emojis. 2018-04-01 23:24:35 -07:00
Rhea Parekh
c650b8fa3e slack importer: Add zerver_realmemoji. 2018-04-01 23:24:35 -07:00
Rhea Parekh
b133d175a7 slack importer: Change 'get_user_data' function implementation.
Change 'get_user_data' function to a more general function
to get data from the slack api using legacy tokens.

Also, change the error handling as upon invalid token,
the response is 200, but the response has an error
field in it.

For eg. Go to the following link with invalid token:
https://slack.com/api/emoji.list?token=xoxp-249056023425
2018-04-01 23:24:35 -07:00
Rhea Parekh
220ad6a386 slack importer: Map standard reactions.
As mentioned in
https://get.slack.help/hc/en-us/articles/202931348-Use-emoji-and-emoticons,
slack supports the standard emoji codes
(https://www.webpagefx.com/tools/emoji-cheat-sheet/)
and majority of them are already supported in Zulip.
2018-04-01 23:24:35 -07:00
Rhea Parekh
8a028142d8 slack importer: Remove id allocation function and its implementation.
Remove allocation ID function from slack import script. All the IDs
count will start from 0. Hence the ID List returned
by the allocation function is of no use, and we remove its implementation.
(example: get_total_messages_and_attachments function is of no use anymore,
hence we remove it)
2018-04-01 23:10:55 -07:00
Puneeth Chaganti
4ce8f2aaa2 upload: Rename upload_message_image to upload_message_file.
Tweaked by tabbott to also fix a Slack import comment.
2018-03-30 13:38:31 -07:00
Rhea Parekh
d0355f52cb slack importer: Show when files are being downloaded. 2018-03-20 14:42:26 -07:00
Rhea Parekh
e7291148e8 slack importer: Add '/me' in content for specific subtypes. 2018-03-20 14:42:26 -07:00
Tim Abbott
99bae7fa5d slack import: Improve algorithm and formatting for attachments.
The previous system would crash with some files (because for some
reason the comment count was 1 but there was no "initial comment") and
also the file comment and file name were sorta redundant.
2018-03-16 11:12:58 -07:00
Rhea Parekh
06071166c7 slack importer: Handle case where attachment download link is not from slack. 2018-03-16 11:12:58 -07:00
Rhea Parekh
b7d6608ba6 slack importer: Clean 'create_converted_data_files' function.
The 'make_new_dir' bool value was used to create a new directory
every time True is passed. Now that avatars and uploads directory
are being created seperately, we don't need this anymore.
2018-03-16 11:12:58 -07:00
Rhea Parekh
4b66a2d0dc slack importer: Add function to fetch and save uploads. 2018-03-16 11:12:58 -07:00
Rhea Parekh
e62945eb86 slack importer: Implement changes in script due to zerver_attachment. 2018-03-16 11:12:58 -07:00
Rhea Parekh
8e2d930644 slack importer: Implement changes in script due to user upload object. 2018-03-16 11:12:58 -07:00
Rhea Parekh
b0851eb20b slack importer: Add helper functions to build attachment object. 2018-03-16 11:12:58 -07:00
Rhea Parekh
68af6e4b7a slack importer: Add helper functions to build user uploads object. 2018-03-16 11:12:58 -07:00
Rhea Parekh
6d85f8c9ef slack importer: Randomize stream colors. 2018-03-15 23:50:32 -07:00
Rhea Parekh
90a3ffc5c0 slack importer: Include only slack's purpose field in description. 2018-03-15 23:50:32 -07:00
Rhea Parekh
8a4f307c43 slack importer: Change topic for imported content. 2018-03-15 23:50:32 -07:00
Tim Abbott
ec21997bfe slack import: Fix importing of bot users from Slack. 2018-03-15 18:35:40 -07:00
Rhea Parekh
a5b0957e5d slack importer: Set domain name in 'do_convert_data'.
The domain name is being set in the helper function
'slack_workspace_to_realm', but it should be set in the main function
'do_convert_data', as we need it in other child functions of
'do_convert_data'.
2018-03-15 18:34:51 -07:00
Rhea Parekh
0d788823c2 slack importer: realm_id cleanup. 2018-03-07 14:07:24 -08:00
Rhea Parekh
7878cc53a3 slack importer: Cleanup build_subscription. 2018-03-07 14:07:24 -08:00
Rhea Parekh
f947194e4c slack importer: Cleanup build_avatar. 2018-03-07 14:07:24 -08:00
Rhea Parekh
5efe05d5b6 slack importer: Cleanup build_zerver_usermessage. 2018-03-07 14:07:24 -08:00
Rhea Parekh
6c4eef16e9 slack importer: Sort messages in the order of their dates.
Sort the list of all messages in the order of their timestamp.
2018-03-06 14:07:09 -08:00