Commit Graph

1937 Commits

Author SHA1 Message Date
Umair Khan
ddaf1a2b67 Return unformatted list from get_language_list. 2016-08-10 16:19:28 -07:00
Steve Howell
06b0df5efc export: Remove spurious select_related() call for Client. 2016-08-10 14:16:17 -07:00
Steve Howell
cb59a11f0a export: Extract get_primary_ids(). 2016-08-10 14:16:17 -07:00
Steve Howell
90e9083b81 export: Extract filter_by_realm(). 2016-08-10 14:16:17 -07:00
Steve Howell
4b6b1b8ad4 export: Extract filter_by_users(). 2016-08-10 14:16:17 -07:00
Steve Howell
db9edfce34 export: Use DATE_FIELDS in fix_datetime_fields().
Now we only call this once per table and use DATE_FIELDS to
look up the data fields.
2016-08-10 14:16:17 -07:00
Steve Howell
35c59fc4d7 export: Clean up export_messages().
This is pretty minor cleanup, but it makes it a little more
explicit what we're writing to the shard file, and it allows
us to use a more specific mypy type when calling
floatify_datetime_fields.
2016-08-10 14:16:17 -07:00
Steve Howell
1d1f36c0b8 export: Always use subprocesses to export UserMessage.
We no longer have an in-process code path to export
UserMessage rows.  We want to only maintain the
subprocess code, which we'll always use in production,
and which will work fine in dev.
2016-08-10 14:16:17 -07:00
Steve Howell
78bbefbf94 export: Create import_attachments. 2016-08-10 14:16:17 -07:00
Rag Sagar
2fef36f15a Add realm-level default language setting.
Adds a new field default language in the zerver_realm model.
This realm level default language will be used as default language
for newly created users. Realm level default language can be
changed from the administration page.

Fixes #1372.
2016-08-09 17:38:29 -07:00
Steve Howell
7ec6a394fe export: Filter Attachment objects by realm. 2016-08-09 16:47:14 -07:00
Steve Howell
cecfaa7761 export: Extract import_message_data(). 2016-08-09 16:47:14 -07:00
Steve Howell
5386ed280e export: Extract update_id_map().
We also use a vanilla dictionary instead of a defaultdict, so
that we explicitly initialize what tables are being re-mapped.
2016-08-09 16:47:14 -07:00
Steve Howell
217ef8a4d2 export: Split fix_foreign_keys() into two functions.
We now have convert_to_id_fields for the simple case, and
re_map_foreign_keys for the more complex case. I also
renamed some parameters and variables.
2016-08-09 16:47:14 -07:00
Rag Sagar
5771e2eb25 Move i18n functions from zerver/views/__init__.py to zerver/lib/i18n.py 2016-08-09 16:10:24 -07:00
Steve Howell
dd88ffccfd export: Extract make_raw() in lib/export.py. 2016-08-09 15:58:27 -07:00
Steve Howell
09fa343bdd export: Use DATE_FIELDS in floatify_datetime_fields.
This avoids a little bit of code duplication, plus it should
make it a little easier to add new date fields in the future.
2016-08-09 15:58:27 -07:00
Steve Howell
c14ab3c91f export: Add annotations to zerver/lib/export.py.
I also fixed some small things like removing unnecessary return
statements, and adding a TODO.

In some cases I explicitly cast stuff at run-time to set() or
str() to appease mypy, as well as make it clear to somebody
reading the code that the callee might not respect ordering
or tolerate unicode.
2016-08-09 15:58:27 -07:00
Steve Howell
f18cc4ae3a export: Added export_avatars_local_helper(). 2016-08-09 15:58:27 -07:00
Umair Khan
acd76eb604 Annotate push_notifications.py. 2016-08-08 15:17:02 -07:00
Umair Khan
0dd7d0dda1 push_notifications: Change name to 'identifier'. 2016-08-08 15:15:29 -07:00
Umair Khan
25dd1aeb5c Annotate zerver/lib/actions.py. 2016-08-08 15:13:29 -07:00
Umair Khan
e34b7a0691 Fix type annotation in message_cache_items. 2016-08-08 15:13:29 -07:00
Tim Abbott
6264ff7039 Add new Zulip realm import/export tool.
The previous export tool would only work properly for small realms,
and was missing a number of important features:
* Export of avatars and uploads from S3
* Export of presence data, activity data, etc.
* Faithful export/import of timestamps
* Parallel export of messages
* Not OOM killing for large realms

The new tool runs as a pair of documented management commands, and
solves all of those problems.

Also we add a new management command for exporting the data of an
individual user.
2016-08-08 14:58:18 -07:00
Tim Abbott
e7adaf8db1 utils: Add mkdir_p implementation. 2016-08-08 14:36:50 -07:00
Umair Khan
d33fc0046f Add APNS feedback server tests. 2016-08-05 10:36:57 -07:00
Umair Khan
40054ce26c Add APNS response listener test case. 2016-08-05 10:36:57 -07:00
Umair Khan
882bb5558b Push to APNS in a loop. 2016-08-05 10:36:57 -07:00
Umair Khan
4e8ca0a326 Make sure apns response code is an integer. 2016-08-05 10:36:57 -07:00
Umair Khan
5c41eae7ec Don't send connection to apns response listener. 2016-08-05 10:36:57 -07:00
Umair Khan
ffaf7cb2ba Create get_connection function for APNS. 2016-08-05 10:36:57 -07:00
Umair Khan
1d2bdfdfb1 Save user id instead of user profile. 2016-08-05 10:36:57 -07:00
Umair Khan
0cac1aa135 For APNS frame identifier generate random int. 2016-08-05 10:36:57 -07:00
Umair Khan
0b5dc56b8d Redis key expiry should be an integer. 2016-08-05 10:36:57 -07:00
Umair Khan
cbd53fbac8 Don't import OrderedDict. 2016-08-05 10:36:57 -07:00
Umair Khan
0b88deb640 APNS_SANDBOX should be a bool.
Fixes: #1480
2016-08-05 10:36:57 -07:00
Tim Abbott
2f13b0b18a actions: Fix various string annotations. 2016-08-04 15:57:03 -07:00
Tim Abbott
d7d8d6b4d2 parallel: Add missing annotation. 2016-08-04 15:57:03 -07:00
Tim Abbott
5253c044e7 test_helpers: Add some missing annotations. 2016-08-04 15:57:03 -07:00
Tim Abbott
0689485666 Add lint check for malformed type annotations missing :.
Also fix the 2 annotations that weren't being checked because of this.
2016-08-04 15:53:23 -07:00
Tim Abbott
028efdcb78 integrations: Add detailed module docstrings. 2016-08-04 12:13:25 -07:00
Tomasz Kolek
0e33b8bd4d Add integrations library.
Define Integration and WebhookIntegration classes.
Change webhook part of integration's guide.
Replace hardcoded webhook urls to generating
based on WEBHOOKS list.
2016-08-04 11:39:20 -07:00
Rishi Gupta
948ea7663c ModelReprMixin: Fix handling of missing __unicode__ function.
The old behavior was to raise an exception, but Django was catching
the exception and doing unexpected things. For instance, in the
manage.py shell, printing out a ModelReprMixin object (with
__unicode__ not implemented) would result in nothing being printed,
rather than it raising a error or otherwise alerting the programmer as
to what was going on.
2016-08-02 11:46:01 -07:00
Umair Khan
2c07f1b19a Use NOREPLY_EMAIL_ADDRESS if email gateway not enabled.
This fixes a regression where missed message emails would not be sent
at all in the event that EMAIL_GATEWAY_PATTERN was unset.

The overall experience still isn't great, but it's better than crashing.

Fixes: #1411

[commit message expanded by tabbott]
2016-07-31 20:38:18 -07:00
Steve Howell
2a37dafcbb Write untested_url_report.txt. 2016-07-29 21:26:51 -07:00
Steve Howell
8b13530712 Add tools/analyze-url-coverage 2016-07-29 21:23:33 -07:00
Steve Howell
21f83afe3a Add --url-coverage option to ./tools/test-backend. 2016-07-29 21:23:33 -07:00
Steve Howell
9fb071947d Remove expected_run_time from @slow test decorator. 2016-07-29 16:41:21 -07:00
Steve Howell
438a118ea5 Simplify enforce_timely_test_completion().
This will lead to minor differences in the warnings that
people see when they run tests that are slow.  We call out
the slowness a little more clearly from a visual standpoint,
and we simplify the calculation of the slowness threshold.

We still allow more time for tests with the `@slow` decorator
to run, but we don't use their expected_run_time.
2016-07-29 16:41:21 -07:00
Steve Howell
4556bf528f Add --report-slow-tests option to tools/test-backend. 2016-07-29 16:41:20 -07:00