Commit Graph

273 Commits

Author SHA1 Message Date
Steve Howell
4a8c70593f populate_db: Add random names for large user batches.
If extra_users > 1000, add some names that are more
interesing than Extra111 User.
2019-12-28 10:56:03 -08:00
Rishi Gupta
e10361a832 models: Replace is_guest and is_realm_admin with UserProfile.role.
This new data model will be more extensible for future work on
features like a primary administrator.
2019-10-06 16:24:37 -07:00
Mateusz Mandera
dbe508bb91 models: Migration of Message.pub_date to date_sent, part 2.
Fixes #1727.

With the server down, apply migrations 0245 and 0246. 0246 will remove
the pub_date column, so it's essential that the previous migrations
ran correctly to copy data before running this.
2019-10-05 19:01:34 -07:00
Mateusz Mandera
4166c901ef do_update_user_custom_profile_data: Rename to ..._if_changed.
This adds clarity to the fact that the function no longer does
anything if the field values haven't changed.
2019-10-01 13:52:43 -07:00
Yashashvi Dave
e05429b6a9 custom fields: Add default external account custom fields. 2019-08-28 15:35:53 -07:00
Mateusz Mandera
16479fa606 populate_db: Extract internal realm creation into a function. 2019-08-23 12:57:24 -07:00
Mateusz Mandera
8e0b6fa6cf populate_db: Use separate realm for system bots, like in production.
This commit alone breaks things, needs to be merged with the follow-up
ones.

welcome-bot is removed from the explicit list, because it already is in
settings.INTERNAL_BOTS.
2019-07-31 15:58:39 -07:00
Anders Kaseorg
fd7803e7f4 settings: Unset STATIC_ROOT in development.
Django’s default FileSystemFinder disallows STATICFILES_DIRS from
containing STATIC_ROOT (by raising an ImproperlyConfigured exception),
because STATIC_ROOT is supposed to be the result of collecting all the
static files in the project, not one of the potentially many sources
of static files.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-24 17:40:31 -07:00
Mateusz Mandera
34b9a7bcad populate_db: Refactor and rename send_messages().
We rename the send_messages function to generate_and_messages, and
factor out the actual sending part of it into a separate function, which
now gets the name send_messages().
2019-07-22 15:56:17 -07:00
Mateusz Mandera
e5138c38b2 populate_db: Add a batching option. 2019-07-22 15:55:28 -07:00
Mateusz Mandera
d1bab42da6 populate_db: Send generated messages in one do_send_messages call.
Sending messages one-by-one is significantly slower compared to taking
advantage of the batch-handling code in do_send_messages, so we pass all
the messages to the function in one go. This is memory-hungry if there's
a lot of messages, so we will allow splitting into smaller batches in
the next commit.
2019-07-22 15:55:28 -07:00
Mateusz Mandera
a50524762c populate_db: Refactor message.pub_date generating code.
The code generating pub_dates for messages would fail to distribute them
across days if tot_messages was too large.
We refactor this code as a separate function (for clarity and to unit
test for the bug we're fixing), and change the structure and naming to a
form that more clearly describes what's happening. We also shift away
from the approach of all the float-to-int conversions as this is in
general tricky and bug prone - django's timedelta() handles floats as
arguments, so we take advantage of that.
2019-07-22 15:55:28 -07:00
Wyatt Hoodes
9b69c1f7fe Merge pull request #12833 from whoodes/clean-up-new-user-bot
bots: Replace remnants of new-user-bot with default-bot@zulip.com.
2019-07-20 14:38:05 -07:00
Mateusz Mandera
c1a8ba2974 populate_db: Use do_change_is_admin for setting api_super_user. 2019-07-15 12:32:29 -07:00
Yashashvi Dave
d7ee2aced1 models: Add external_account in custom profile field types.
Add new custom profile field type, External account.
External account field links user's social media
profile with account. e.g. GitHub, Twitter, etc.

Fixes part of #12302
2019-07-09 17:21:54 -07:00
Yashashvi Dave
f813fdcd5e populate_db: Rename URL type custom profile field.
Rename URL type custom profile field in populate db to avoid confusion
with the "GitHub profile" custom external account profile field we'll
be adding shortly.
2019-07-09 17:21:39 -07:00
Vaibhav
fa85b92eda populate_db: Add MD content to stream descriptions and user bio.
This adds a 'bold' element to `all` stream and 'code' element to
`test` stream in populate_db. Also changes the bio for Hamlet to
include a list.
2019-06-24 22:10:24 -07:00
Wyatt Hoodes
fb58b54293 test-backend: Move test_messages.json to var/<uuid>/test-backend. 2019-06-24 12:23:09 -07:00
Tim Abbott
12756b48a0 populate_db: Fix help text for --test-suite. 2019-02-20 17:42:30 -08:00
Anders Kaseorg
f5197518a9 analytics/zilencer/zproject: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:31:45 -08:00
Vishnu Ks
2889de6d90 populate_db: Extract subscribe_users_to_streams function. 2018-11-29 12:15:58 -08:00
Tim Abbott
9c224ccdd3 populate_db: Suppress logging output for push notifications.
This fixes a lot of spammy output of the form:

2018-11-27 17:46:48.279 INFO [zerver.lib.push_notifications] Sending push notification to user 46

when running populate_db, which is both confusing (since we're not
actually sending push notifications here) and spammy.
2018-11-27 09:50:17 -08:00
Yago González
6a192ac84c utils: Move random API key generator as generate_api_key.
random_api_key, the function we use to generate random tokens for API
keys, has been moved to zerver/lib/utils.py because it's used in more
parts of the codebase (apart from user creation), and having it in
zerver/lib/create_user.py was prone to cyclic dependencies.

The function has also been renamed to generate_api_key to have an
imperative name, that makes clearer what it does.
2018-08-08 16:45:25 -07:00
Roman Godov
c0806917ec models: Rename Realm.restricted_to_domain field.
This renames Realm.restricted_to_domain field to
emails_restricted_to_domains, for greater clarity as to what it does
just from seeing the setting name, without having to look it up.

Fixes part of #10042.
2018-07-31 09:28:33 -07:00
Vishnu Ks
ff4c1ca2c1 models: Add SUBSCRIPTION_CREATED event type constant to RealmAuditLog. 2018-07-10 15:42:26 +05:30
Shubham Dhama
7cbacdb8c2 populate_db: Integrate populate_analytics_db.
As detailed in the documentation changes, this simplifies the
development workflow for doing UI work on the /stats pages.

The cost is a ~10% increase the time it takes to run `populate_db`,
which doesn't happen very often (and for most purposes manifests as a
1% increase in the time it takes to rebuild the database from scratch).
2018-07-09 15:05:48 +05:30
Yashashvi Dave
8909cb1d15 custom fields: Allow list of users in user type of custom fields.
Allow user to add more than one user-value in user type of custom
fields.

Tweaked by tabbott to improve the models.py code and type annotations.
2018-06-16 09:37:49 -07:00
Tim Abbott
27e7770540 populate_db: Initialize the outgoing webhook bot properly.
Previously, we created the bot, but didn't initialize its service
object.
2018-06-01 08:48:17 -07:00
Robert Hönig
497cd4a1f9 outgoing webhooks: Change default URL in all tests.
This reflects the changes to the default URL publicly
displayed to the user. It also changes the default
URL of the default test server outgoing webhook, which
prevented the test server flaskbotrc from working out
of the box.
2018-05-28 10:40:44 -07:00
Yashashvi Dave
e82c879b85 custom fields: Add user type of custom fields.
Fixes #8878
2018-05-27 23:01:21 -07:00
Aditya Bansal
8cfb437677 provision: Don't run create_realm_internal_bots management command.
We essentially stop running create_realm_internal_bots during
every provisioing and move its operations to run from populate db.
In fact to speed things up a bit we actually make populate db call the
funcs which create_realm_internal_bots calls behind the scenes.

Fixes: #9467.
2018-05-23 13:34:52 +05:30
Tim Abbott
1b47b2e7cd populate_db: Fix an out-of-date mypy annotation. 2018-05-22 14:46:03 -07:00
Tim Abbott
8a64d8ef06 populate_db: Fix stream colors being generated wrong.
Previously, the stream colors index i was accidentally a function only
of the user, so each user got the same color for all their streams.

This should provide a lot nicer-looking development environment
experience.
2018-05-22 14:29:43 -07:00
Tim Abbott
4c8a69e3fb populate_db: Add an initial stream for the default guest user. 2018-05-22 14:29:43 -07:00
Sampriti Panda
4ce5a85a49 populate_db: Make test_suite data generation more deterministic. 2018-05-22 14:07:55 -07:00
Shubham Padia
4d70d032bc populate_db: Make is_web_public default to False.
Explicit mentions of `is_web_public` as False in the stream dict to be
passed to `bulk_create_streams` have been removed.
2018-05-16 13:40:22 -07:00
Shubham Padia
315047a38b populate_db: Make invite_only default to False.
Explicit mentions of `invite_only` as False in the stream dict to be
passed to `bulk_create_streams` have been removed.
2018-05-16 13:40:10 -07:00
Shubham Padia
7b4f6e0058 tests: Make is_announcement_only true for announce stream in populate_db.
Makes announce stream `is_announcement_only` for the dev db for easier
manual testing. The default value for `is_announcement_only` in
`bulk_create_streams` is False.
2018-05-16 13:35:45 -07:00
Aditya Bansal
67bf71472a zilencer: Change use of typing.Text to str. 2018-05-12 15:22:39 -07:00
Aditya Bansal
9629be689b populate_db: Add a web public stream to dev database.
We flip the Stream "Rome" to be a web public stream. Also we add
attribute is_web_public in various stream dicts and in the
bulk_create_streams function of bulk_create.py responsible for
default stream creation in dev environment.
2018-05-02 15:23:33 -07:00
Yashashvi Dave
0d7d94d0db custom fields: Add support for custom URL field type. 2018-04-30 10:53:23 -07:00
Yashashvi Dave
4033f210af custom fields: Add support for custom date field type. 2018-04-30 23:04:25 +05:30
Tim Abbott
8344bd171e populate_db: Properly initialize the email_gateway_bot.
Apparently, this bot account was not properly being tagged as an API
super user in the test database; resulting in incorrect behavior if we
tried to send to a private stream in a test.

(Note that there seems to also be a similar issue in production, that
we don't understand the cause of; that is unrelated).
2018-04-27 13:56:04 -07:00
Umair Khan
cf2f6b38dd profile: Add choice field.
Fixes part of #8878
2018-04-26 00:35:51 -07:00
Shubham Dhama
26d2ffa821 populate_db: Add new user of "guest" type.
The purpose of this user is to act as a guest.
(This is a preliminary step in adding the guest type of user
and is a part of #8385.)
2018-04-20 16:20:00 -07:00
Preston Hansen
76d6c71595 tests: Move zerver/fixtures to zerver/tests/fixtures for clarity.
Fixes #9153.
2018-04-19 21:50:17 -07:00
Umair Khan
0420b89468 profile: Add hint attribute to custom profile fields.
This is the model for #8876.
2018-04-19 11:32:35 -07:00
Umair Khan
c30a282dd9 profile: Remove integer and float fields. 2018-04-02 09:46:21 -07:00
sinwar
79672fffc6 populate_db: Pre-populate open graph cache to avoid network activity.
This prevents annoying network errors when trying to populate the
database without Internet access.

Cleaned up a bit by tabbott.

Fixes #8289.
2018-03-27 16:42:59 -07:00
YJDave
6ac687790c populate_db: Remove the test_suite check for custom profile fields.
To ensure that we have some basic data for custom profile settings,
in the `populate_db` data set, remove `options['test_suite']` check
for adding intial custom profile data.
2018-03-21 16:05:31 -07:00