diff --git a/docs/subsystems/auth.md b/docs/development/authentication.md similarity index 100% rename from docs/subsystems/auth.md rename to docs/development/authentication.md diff --git a/docs/development/index.rst b/docs/development/index.rst index 21034672ad..23b1bbe3a5 100644 --- a/docs/development/index.rst +++ b/docs/development/index.rst @@ -10,3 +10,4 @@ Development Environment Advanced Setup (non-Vagrant) Using the development environment Developing remotely + Authentication in the development environment diff --git a/docs/production/authentication-methods.md b/docs/production/authentication-methods.md index 8e46153764..6e30714b14 100644 --- a/docs/production/authentication-methods.md +++ b/docs/production/authentication-methods.md @@ -489,7 +489,7 @@ For example, the was about 30 lines of code, plus some documentation and an [automatically generated migration][schema-migrations]. We also have helpful developer documentation on -[testing auth backends](../subsystems/auth.md). +[testing auth backends](../development/authentication.md). [schema-migrations]: ../subsystems/schema-migrations.md [python-social-auth]: https://python-social-auth.readthedocs.io/en/latest/ diff --git a/docs/subsystems/index.rst b/docs/subsystems/index.rst index 0c9f560ffd..ec341cb1c7 100644 --- a/docs/subsystems/index.rst +++ b/docs/subsystems/index.rst @@ -18,12 +18,10 @@ Subsystems Documentation realms management-commands schema-migrations - migration-renumbering hashchange-system emoji hotspots full-text-search - auth email analytics client diff --git a/docs/subsystems/migration-renumbering.md b/docs/subsystems/migration-renumbering.md deleted file mode 100644 index 7fabdd52d3..0000000000 --- a/docs/subsystems/migration-renumbering.md +++ /dev/null @@ -1,305 +0,0 @@ -# Renumbering Migrations -When you rebase your development branch off of -a newer copy of master, and your branch contains -new database migrations, you can occasionally get -thrown off by conflicting migrations that are -new in master. - -To help you understand how to deal with these -conflicts, I am about to narrate an exercise -where I bring my development branch called -showell-topic up to date with master. - -**Note:** You can also use the command `./tools/renumber-migrations` to -automatically perform migration renumbering. - -In this example, -there is a migration on master called -`0024_realm_allow_message_editing.py`, and -that was the most recent migration at the -time I started working on my branch. In -my branch I created migrations 0025 and 0026, -but then meanwhile on master somebody else -created their own migration 0025. - -Anyway, on with the details... - -First, I go to showell-topic and run tests to -make sure that I'm starting with a clean, albeit -out-of-date, dev branch: - -``` -showell@Steves-MBP ~/zulip (showell-approximate) $ git checkout showell-topic -Switched to branch 'showell-topic' - -showell@Steves-MBP ~/zulip (showell-topic) $ git status -# On branch showell-topic -nothing to commit, working directory clean - -(zulip-py3-venv) vagrant@ubuntu-bionic:/srv/zulip$ ./tools/test-backend - -DONE! -``` - -Next, I fetch changes from upstream: - -``` -showell@Steves-MBP ~/zulip (showell-topic) $ git checkout master -Switched to branch 'master' - -showell@Steves-MBP ~/zulip (master) $ git fetch upstream - -showell@Steves-MBP ~/zulip (master) $ git merge upstream/master -Updating 2967341..09754c9 -Fast-forward - -``` - -Then I go back to showell-topic: - -``` -showell@Steves-MBP ~/zulip (master) $ git checkout showell-topic -Switched to branch 'showell-topic' -``` - -You may want to make note of your HEAD commit on your branch -before you start rebasing, in case you need to start over, or -do like I do and rely on being able to find it via github. I'm -not showing the details of that, since people have different -styles for managing botched rebases. - -Anyway, I rebase to master as follows: - -``` -showell@Steves-MBP ~/zulip (showell-topic) $ git rebase -i master -Successfully rebased and updated refs/heads/showell-topic. -``` - -Note that my rebase was conflict-free from git's point of view, -but I still need to run the tests to make sure there weren't any -semantic conflicts with the new changes from master: - -``` -(zulip-py3-venv) vagrant@ubuntu-bionic:/srv/zulip$ ./tools/test-backend - - - - File "/srv/zulip-venv-cache/ad3a375e95a56d911510d7edba7e17280d227bc7/zulip-venv/local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 105, in handle - "'./manage.py makemigrations --merge'" % name_str -django.core.management.base.CommandError: Conflicting migrations detected (0026_topics_backfill, 0025_realm_message_content_edit_limit in zerver). -To fix them run './manage.py makemigrations --merge' - - - - File "/srv/zulip/zerver/lib/db.py", line 33, in execute - return wrapper_execute(self, super().execute, query, vars) - File "/srv/zulip/zerver/lib/db.py", line 20, in wrapper_execute - return action(sql, params) -django.db.utils.ProgrammingError: relation "zerver_realmfilter" does not exist -LINE 1: ...n", "zerver_realmfilter"."url_format_string" FROM "zerver_re... -``` - -The above traceback is fairly noisy, but it's pretty apparent that -I have migrations that are out of order. More precisely, my 0025 migration -points to 0024 as its dependency, where it really should point to the -other 0025 as its dependency, and I need to renumber my migrations to -0026 and 0027. - -Let's take a peek at the migrations directory: - -``` -showell@Steves-MBP ~/zulip (showell-topic) $ ls -r zerver/migrations/*.py | head -6 -zerver/migrations/__init__.py -zerver/migrations/0026_topics_backfill.py -zerver/migrations/0025_realm_message_content_edit_limit.py -zerver/migrations/0025_add_topic_table.py -zerver/migrations/0024_realm_allow_message_editing.py -zerver/migrations/0023_userprofile_default_language.py -``` - -We have two different 0025 migrations that both depend on 0024: - -``` -showell@Steves-MBP ~/zulip (showell-topic) $ grep -B 1 0024 zerver/migrations/*.py -zerver/migrations/0025_add_topic_table.py- dependencies = [ -zerver/migrations/0025_add_topic_table.py: ('zerver', '0024_realm_allow_message_editing'), --- -zerver/migrations/0025_realm_message_content_edit_limit.py- dependencies = [ -zerver/migrations/0025_realm_message_content_edit_limit.py: ('zerver', '0024_realm_allow_message_editing'), -``` - -I will now start the process of renaming `0025_add_topic_table.py` to -be `0026_add_topic_table.py` and having it depend on -`0025_realm_message_content_edit_limit`. Before I start, I want to -know which of my commits created my 0025 migration: - -``` -showell@Steves-MBP ~/zulip (showell-topic) $ git log --pretty=oneline zerver/migrations/0025_add_topic_table.py -d859e6ffc165e822cec39152a5814ca7ce94d172 Add Topic and Message.topic to models -``` - -Here is the transcript, and hopefully what I did inside of vim is apparent -from the diff: - -``` -showell@Steves-MBP ~/zulip (showell-topic) $ cd zerver/migrations/ -showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic) $ git mv 0025_add_topic_table.py 0026_add_topic_table.py -showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic +) $ vim 0026_add_topic_table.py -showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic *+) $ git diff -diff --git a/zerver/migrations/0026_add_topic_table.py b/zerver/migrations/0026_add_topic_table.py -index 2c8c07a..43351eb 100644 ---- a/zerver/migrations/0026_add_topic_table.py -+++ b/zerver/migrations/0026_add_topic_table.py -@@ -8,7 +8,7 @@ from django.db import migrations, models - class Migration(migrations.Migration): - - dependencies = [ -- ('zerver', '0024_realm_allow_message_editing'), -+ ('zerver', '0025_realm_message_content_edit_limit'), - ] - - operations = [ -showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic *+) $ git commit -am 'temp rename migration' -[showell-topic 45cf5e9] temp rename migration - 1 file changed, 1 insertion(+), 1 deletion(-) - rename zerver/migrations/{0025_add_topic_table.py => 0026_add_topic_table.py} (93%) -showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic) $ git status -# On branch showell-topic -nothing to commit, working directory clean -``` - -Next, I want to rewrite the history of my branch. When I'm in the -interactive rebase (not shown), I need to make the temp commit be a -"fix" for the original commit that I noted above: - -``` -showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic) $ git rebase -i master -[detached HEAD c1f2e69] Add Topic and Message.topic to models - 2 files changed, 38 insertions(+) - create mode 100644 zerver/migrations/0026_add_topic_table.py -Successfully rebased and updated refs/heads/showell-topic. -``` - -I did this to verify that I rebased correctly: - -``` -showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic) $ git log 0026_add_topic_table.py -commit c1f2e69e716beb4031c628fe2189b49f04770d03 -Author: Steve Howell -Date: Thu Jul 14 13:26:15 2016 -0700 - - Add Topic and Message.topic to models -showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic) $ git show c1f2e69e716beb4031c628fe2189b49f04770d03 - -``` - -Next, I follow a very similar process for my second migration-related -commit on my branch: - - -``` -showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic) $ git log --pretty=oneline 0026_topics_backfill.py -ba43e1ffb072f4e6a66ffb5c4030ff3a17d53792 (unfinished) stub commit for topic backfill -showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic) $ git mv 0026_topics_backfill.py 0027_topics_backfill.py -showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic +) $ vim 0027_topics_backfill.py -showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic *+) $ git diff -diff --git a/zerver/migrations/0027_topics_backfill.py b/zerver/migrations/0027_topics_backfill.py -index 766b075..05ea2bb 100644 ---- a/zerver/migrations/0027_topics_backfill.py -+++ b/zerver/migrations/0027_topics_backfill.py -@@ -8,7 +8,7 @@ from django.db import migrations, models - class Migration(migrations.Migration): - - dependencies = [ -- ('zerver', '0025_add_topic_table'), -+ ('zerver', '0026_add_topic_table'), - ] - - operations = [ -showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic *+) $ git commit -am 'temp rename migration' -[showell-topic 02ef15b] temp rename migration - 1 file changed, 1 insertion(+), 1 deletion(-) - rename zerver/migrations/{0026_topics_backfill.py => 0027_topics_backfill.py} (90%) -showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic) $ git rebase -i master -[detached HEAD 8022839] (unfinished) stub commit for topic backfill - 1 file changed, 20 insertions(+) - create mode 100644 zerver/migrations/0027_topics_backfill.py -Successfully rebased and updated refs/heads/showell-topic. -``` - -My rebase looked something like this after I edited the commits: - -``` - 1 pick eee291d Add subject_topic_awareness() test helper. - 2 pick c1f2e69 Add Topic and Message.topic to models - 3 pick 46c04b5 Call new update_topic() in pre_save_message(). - 4 pick df20dc8 Write to Topic table for topic edits. - 5 pick ba43e1f (unfinished) stub commit for topic backfill - 6 f 02ef15b temp rename migration - 7 pick b8e93d2 Add CATCH_TOPIC_MIGRATION_BUGS. - 8 pick 644ccae Have get_context_for_message use topic_id. - 9 pick 6303f5b Have update_message_flags user topic_id. - 10 pick 9f9da5a Have narrowing searches use topic id. - 11 pick 0b37ef7 Assert that new_topic=True obliterates message.subject - 12 pick dff8eee Use new_topics=True in test_bulk_message_fetching(). - 13 pick bc377a0 Use topic_id when propagating message edits. - 14 pick 11fde9c Have message cache use Topic table (and more...). - 15 pick 518649b Use topic_name() in to_log_dict(). - 16 - 17 # Rebase 09754c9..02ef15b onto 09754c9 - 18 # - 19 # Commands: - 20 # p, pick = use commit - 21 # r, reword = use commit, but edit the commit message - 22 # e, edit = use commit, but stop for amending - 23 # s, squash = use commit, but meld into previous commit - 24 # f, fixup = like "squash", but discard this commit's log message - 25 # x, exec = run command (the rest of the line) using shell -``` - -I double check that everything went fine: - -``` -showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic) $ git log 0027_topics_backfill.py -commit 8022839f9168e643ae08365bfb40f1de2e64d426 -Author: Steve Howell -Date: Thu Jul 14 15:51:06 2016 -0700 - - (unfinished) stub commit for topic backfill -showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic) $ git show 8022839f9168e643ae08365bfb40f1de2e64d426 - -``` - -And then I run the tests and cross my fingers!!!: - -``` -(zulip-py3-venv) vagrant@ubuntu-bionic:/srv/zulip$ ./tools/test-backend - - Applying zerver.0023_userprofile_default_language... OK - Applying zerver.0024_realm_allow_message_editing... OK - Applying zerver.0025_realm_message_content_edit_limit... OK - Applying zerver.0026_add_topic_table... OK - Applying zerver.0027_topics_backfill... OK - Applying zilencer.0001_initial... OK -Successfully populated test database. -DROP DATABASE -CREATE DATABASE -Running zerver.tests.test_auth_backends.AuthBackendTest.test_devauth_backend -Running zerver.tests.test_auth_backends.AuthBackendTest.test_dummy_backend -Running zerver.tests.test_auth_backends.AuthBackendTest.test_email_auth_backend - -FAILED! -``` - -Ugh, my tests still fail due to some non-migration-related changes on master. -The good news, however, is that my migrations are cleaned up. - -So I've shown you the excruciating details of fixing up migrations -in a complicated branch, but let's step back and look at the big -picture. You need to get these things right: - -- Rename the migrations on your branch. -- Fix their dependencies. -- Rewrite your git history so that it appears like you never branched off an old copy of master. - -The hardest part of the process will probably be cleaning up your git history. diff --git a/docs/subsystems/schema-migrations.md b/docs/subsystems/schema-migrations.md index b4ab88c70a..a7b7831d1f 100644 --- a/docs/subsystems/schema-migrations.md +++ b/docs/subsystems/schema-migrations.md @@ -30,7 +30,7 @@ migrations. fix this, you can either run `./tools/renumber-migrations` which renumbers your migration(s) and fixes up the "dependencies" entries in your migration(s), and then rewrite your git history as needed, or you can do it - manually. There is a tutorial [here](migration-renumbering.md) that + manually. There is a tutorial [here](#renumbering-migrations) that walks you though that process. * **Atomicity**. By default, each Django migration is run atomically @@ -103,6 +103,312 @@ few properties: tables are done in large-scale systems, since it ensures that the system can continue running happily during the migration. +## Renumbering Migrations +When you rebase your development branch off of +a newer copy of master, and your branch contains +new database migrations, you can occasionally get +thrown off by conflicting migrations that are +new in master. + +To help you understand how to deal with these +conflicts, I am about to narrate an exercise +where I bring my development branch called +showell-topic up to date with master. + +**Note:** You can also use the command `./tools/renumber-migrations` to +automatically perform migration renumbering. + +In this example, +there is a migration on master called +`0024_realm_allow_message_editing.py`, and +that was the most recent migration at the +time I started working on my branch. In +my branch I created migrations 0025 and 0026, +but then meanwhile on master somebody else +created their own migration 0025. + +Anyway, on with the details... + +First, I go to showell-topic and run tests to +make sure that I'm starting with a clean, albeit +out-of-date, dev branch: + +``` +showell@Steves-MBP ~/zulip (showell-approximate) $ git checkout showell-topic +Switched to branch 'showell-topic' + +showell@Steves-MBP ~/zulip (showell-topic) $ git status +# On branch showell-topic +nothing to commit, working directory clean + +(zulip-py3-venv) vagrant@ubuntu-bionic:/srv/zulip$ ./tools/test-backend + +DONE! +``` + +Next, I fetch changes from upstream: + +``` +showell@Steves-MBP ~/zulip (showell-topic) $ git checkout master +Switched to branch 'master' + +showell@Steves-MBP ~/zulip (master) $ git fetch upstream + +showell@Steves-MBP ~/zulip (master) $ git merge upstream/master +Updating 2967341..09754c9 +Fast-forward + +``` + +Then I go back to showell-topic: + +``` +showell@Steves-MBP ~/zulip (master) $ git checkout showell-topic +Switched to branch 'showell-topic' +``` + +You may want to make note of your HEAD commit on your branch +before you start rebasing, in case you need to start over, or +do like I do and rely on being able to find it via github. I'm +not showing the details of that, since people have different +styles for managing botched rebases. + +Anyway, I rebase to master as follows: + +``` +showell@Steves-MBP ~/zulip (showell-topic) $ git rebase -i master +Successfully rebased and updated refs/heads/showell-topic. +``` + +Note that my rebase was conflict-free from git's point of view, +but I still need to run the tests to make sure there weren't any +semantic conflicts with the new changes from master: + +``` +(zulip-py3-venv) vagrant@ubuntu-bionic:/srv/zulip$ ./tools/test-backend + + + + File "/srv/zulip-venv-cache/ad3a375e95a56d911510d7edba7e17280d227bc7/zulip-venv/local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 105, in handle + "'./manage.py makemigrations --merge'" % name_str +django.core.management.base.CommandError: Conflicting migrations detected (0026_topics_backfill, 0025_realm_message_content_edit_limit in zerver). +To fix them run './manage.py makemigrations --merge' + + + + File "/srv/zulip/zerver/lib/db.py", line 33, in execute + return wrapper_execute(self, super().execute, query, vars) + File "/srv/zulip/zerver/lib/db.py", line 20, in wrapper_execute + return action(sql, params) +django.db.utils.ProgrammingError: relation "zerver_realmfilter" does not exist +LINE 1: ...n", "zerver_realmfilter"."url_format_string" FROM "zerver_re... +``` + +The above traceback is fairly noisy, but it's pretty apparent that +I have migrations that are out of order. More precisely, my 0025 migration +points to 0024 as its dependency, where it really should point to the +other 0025 as its dependency, and I need to renumber my migrations to +0026 and 0027. + +Let's take a peek at the migrations directory: + +``` +showell@Steves-MBP ~/zulip (showell-topic) $ ls -r zerver/migrations/*.py | head -6 +zerver/migrations/__init__.py +zerver/migrations/0026_topics_backfill.py +zerver/migrations/0025_realm_message_content_edit_limit.py +zerver/migrations/0025_add_topic_table.py +zerver/migrations/0024_realm_allow_message_editing.py +zerver/migrations/0023_userprofile_default_language.py +``` + +We have two different 0025 migrations that both depend on 0024: + +``` +showell@Steves-MBP ~/zulip (showell-topic) $ grep -B 1 0024 zerver/migrations/*.py +zerver/migrations/0025_add_topic_table.py- dependencies = [ +zerver/migrations/0025_add_topic_table.py: ('zerver', '0024_realm_allow_message_editing'), +-- +zerver/migrations/0025_realm_message_content_edit_limit.py- dependencies = [ +zerver/migrations/0025_realm_message_content_edit_limit.py: ('zerver', '0024_realm_allow_message_editing'), +``` + +I will now start the process of renaming `0025_add_topic_table.py` to +be `0026_add_topic_table.py` and having it depend on +`0025_realm_message_content_edit_limit`. Before I start, I want to +know which of my commits created my 0025 migration: + +``` +showell@Steves-MBP ~/zulip (showell-topic) $ git log --pretty=oneline zerver/migrations/0025_add_topic_table.py +d859e6ffc165e822cec39152a5814ca7ce94d172 Add Topic and Message.topic to models +``` + +Here is the transcript, and hopefully what I did inside of vim is apparent +from the diff: + +``` +showell@Steves-MBP ~/zulip (showell-topic) $ cd zerver/migrations/ +showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic) $ git mv 0025_add_topic_table.py 0026_add_topic_table.py +showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic +) $ vim 0026_add_topic_table.py +showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic *+) $ git diff +diff --git a/zerver/migrations/0026_add_topic_table.py b/zerver/migrations/0026_add_topic_table.py +index 2c8c07a..43351eb 100644 +--- a/zerver/migrations/0026_add_topic_table.py ++++ b/zerver/migrations/0026_add_topic_table.py +@@ -8,7 +8,7 @@ from django.db import migrations, models + class Migration(migrations.Migration): + + dependencies = [ +- ('zerver', '0024_realm_allow_message_editing'), ++ ('zerver', '0025_realm_message_content_edit_limit'), + ] + + operations = [ +showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic *+) $ git commit -am 'temp rename migration' +[showell-topic 45cf5e9] temp rename migration + 1 file changed, 1 insertion(+), 1 deletion(-) + rename zerver/migrations/{0025_add_topic_table.py => 0026_add_topic_table.py} (93%) +showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic) $ git status +# On branch showell-topic +nothing to commit, working directory clean +``` + +Next, I want to rewrite the history of my branch. When I'm in the +interactive rebase (not shown), I need to make the temp commit be a +"fix" for the original commit that I noted above: + +``` +showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic) $ git rebase -i master +[detached HEAD c1f2e69] Add Topic and Message.topic to models + 2 files changed, 38 insertions(+) + create mode 100644 zerver/migrations/0026_add_topic_table.py +Successfully rebased and updated refs/heads/showell-topic. +``` + +I did this to verify that I rebased correctly: + +``` +showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic) $ git log 0026_add_topic_table.py +commit c1f2e69e716beb4031c628fe2189b49f04770d03 +Author: Steve Howell +Date: Thu Jul 14 13:26:15 2016 -0700 + + Add Topic and Message.topic to models +showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic) $ git show c1f2e69e716beb4031c628fe2189b49f04770d03 + +``` + +Next, I follow a very similar process for my second migration-related +commit on my branch: + + +``` +showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic) $ git log --pretty=oneline 0026_topics_backfill.py +ba43e1ffb072f4e6a66ffb5c4030ff3a17d53792 (unfinished) stub commit for topic backfill +showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic) $ git mv 0026_topics_backfill.py 0027_topics_backfill.py +showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic +) $ vim 0027_topics_backfill.py +showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic *+) $ git diff +diff --git a/zerver/migrations/0027_topics_backfill.py b/zerver/migrations/0027_topics_backfill.py +index 766b075..05ea2bb 100644 +--- a/zerver/migrations/0027_topics_backfill.py ++++ b/zerver/migrations/0027_topics_backfill.py +@@ -8,7 +8,7 @@ from django.db import migrations, models + class Migration(migrations.Migration): + + dependencies = [ +- ('zerver', '0025_add_topic_table'), ++ ('zerver', '0026_add_topic_table'), + ] + + operations = [ +showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic *+) $ git commit -am 'temp rename migration' +[showell-topic 02ef15b] temp rename migration + 1 file changed, 1 insertion(+), 1 deletion(-) + rename zerver/migrations/{0026_topics_backfill.py => 0027_topics_backfill.py} (90%) +showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic) $ git rebase -i master +[detached HEAD 8022839] (unfinished) stub commit for topic backfill + 1 file changed, 20 insertions(+) + create mode 100644 zerver/migrations/0027_topics_backfill.py +Successfully rebased and updated refs/heads/showell-topic. +``` + +My rebase looked something like this after I edited the commits: + +``` + 1 pick eee291d Add subject_topic_awareness() test helper. + 2 pick c1f2e69 Add Topic and Message.topic to models + 3 pick 46c04b5 Call new update_topic() in pre_save_message(). + 4 pick df20dc8 Write to Topic table for topic edits. + 5 pick ba43e1f (unfinished) stub commit for topic backfill + 6 f 02ef15b temp rename migration + 7 pick b8e93d2 Add CATCH_TOPIC_MIGRATION_BUGS. + 8 pick 644ccae Have get_context_for_message use topic_id. + 9 pick 6303f5b Have update_message_flags user topic_id. + 10 pick 9f9da5a Have narrowing searches use topic id. + 11 pick 0b37ef7 Assert that new_topic=True obliterates message.subject + 12 pick dff8eee Use new_topics=True in test_bulk_message_fetching(). + 13 pick bc377a0 Use topic_id when propagating message edits. + 14 pick 11fde9c Have message cache use Topic table (and more...). + 15 pick 518649b Use topic_name() in to_log_dict(). + 16 + 17 # Rebase 09754c9..02ef15b onto 09754c9 + 18 # + 19 # Commands: + 20 # p, pick = use commit + 21 # r, reword = use commit, but edit the commit message + 22 # e, edit = use commit, but stop for amending + 23 # s, squash = use commit, but meld into previous commit + 24 # f, fixup = like "squash", but discard this commit's log message + 25 # x, exec = run command (the rest of the line) using shell +``` + +I double check that everything went fine: + +``` +showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic) $ git log 0027_topics_backfill.py +commit 8022839f9168e643ae08365bfb40f1de2e64d426 +Author: Steve Howell +Date: Thu Jul 14 15:51:06 2016 -0700 + + (unfinished) stub commit for topic backfill +showell@Steves-MBP ~/zulip/zerver/migrations (showell-topic) $ git show 8022839f9168e643ae08365bfb40f1de2e64d426 + +``` + +And then I run the tests and cross my fingers!!!: + +``` +(zulip-py3-venv) vagrant@ubuntu-bionic:/srv/zulip$ ./tools/test-backend + + Applying zerver.0023_userprofile_default_language... OK + Applying zerver.0024_realm_allow_message_editing... OK + Applying zerver.0025_realm_message_content_edit_limit... OK + Applying zerver.0026_add_topic_table... OK + Applying zerver.0027_topics_backfill... OK + Applying zilencer.0001_initial... OK +Successfully populated test database. +DROP DATABASE +CREATE DATABASE +Running zerver.tests.test_auth_backends.AuthBackendTest.test_devauth_backend +Running zerver.tests.test_auth_backends.AuthBackendTest.test_dummy_backend +Running zerver.tests.test_auth_backends.AuthBackendTest.test_email_auth_backend + +FAILED! +``` + +Ugh, my tests still fail due to some non-migration-related changes on master. +The good news, however, is that my migrations are cleaned up. + +So I've shown you the excruciating details of fixing up migrations +in a complicated branch, but let's step back and look at the big +picture. You need to get these things right: + +- Rename the migrations on your branch. +- Fix their dependencies. +- Rewrite your git history so that it appears like you never branched off an old copy of master. + +The hardest part of the process will probably be cleaning up your git history. + ## Automated testing for migrations Zulip has support for writing automated tests for your database diff --git a/templates/zerver/config_error.html b/templates/zerver/config_error.html index e824a9378e..a4a31d6c20 100644 --- a/templates/zerver/config_error.html +++ b/templates/zerver/config_error.html @@ -88,7 +88,7 @@ {% if development_environment %}

See also - the SAML + the SAML guide for the development environment.

{% endif %} diff --git a/templates/zerver/dev_tools.html b/templates/zerver/dev_tools.html index a91becd8e4..eec8477d71 100644 --- a/templates/zerver/dev_tools.html +++ b/templates/zerver/dev_tools.html @@ -103,7 +103,7 @@

We also have - documentation on testing LDAP, Google & GitHub authentication in the development environment. + documentation on testing LDAP, Google & GitHub authentication in the development environment.

diff --git a/tools/linter_lib/custom_check.py b/tools/linter_lib/custom_check.py index 2acaa8643c..d0a23f6a24 100644 --- a/tools/linter_lib/custom_check.py +++ b/tools/linter_lib/custom_check.py @@ -776,7 +776,7 @@ markdown_docs_length_exclude = { "docs/development/setup-vagrant.md", # Have wide output in code blocks "docs/subsystems/logging.md", - "docs/subsystems/migration-renumbering.md", + "docs/subsystems/schema-migrations.md", # Have curl commands with JSON that would be messy to wrap "zerver/webhooks/helloworld/doc.md", "zerver/webhooks/trello/doc.md", diff --git a/zproject/backends.py b/zproject/backends.py index d3d2609d03..d35acd409a 100644 --- a/zproject/backends.py +++ b/zproject/backends.py @@ -2,9 +2,9 @@ # # * https://zulip.readthedocs.io/en/latest/production/authentication-methods.html and # zproject/prod_settings_template.py have user-level configuration documentation. -# * https://zulip.readthedocs.io/en/latest/subsystems/auth.html has developer-level -# documentation, especially on testing authentication backends in the Zulip -# development environment. +# * https://zulip.readthedocs.io/en/latest/development/authentication.html +# has developer-level documentation, especially on testing authentication backends +# in the Zulip development environment. # # Django upstream's documentation for authentication backends is also # helpful background. The most important detail to understand for