ldap: Fix the syncing of user role via AUTH_LDAP_USER_FLAGS_BY_GROUP.

This was broken, due the mechanism simply using our
is_guest/is_realm_admin/etc. role setters, but failing to adjust system
group memberships - resulting in corrupted database state.
We need to ensure that change_user_role is called for setting user role.

There are two relevant codepaths that run the sync based on
AUTH_LDAP_USER_FLAGS_BY_GROUP and thus need to get this right:
1. manage.py sync_ldap_user_data
2. Just-in-time user creation when a user without a Zulip account logs
   in for the first using their ldap credentials. After
   get_or_build_user returns, django-auth-ldap sees that the user
   account has just been created, and proceeds to run ._populate_user().

Now that both user.save() and do_change_user_realm will be getting
called together, we need to ensure this always happens atomically.

This imposes the need to override _get_or_create_user to put it in a
transaction. The troublesome consequence is that this new
`atomic(savepoint=False)` causes the usual type of issue, where tests
testing error get their transaction rolled back and cannot continue
executing.

To get around that, we add a test helper
`artificial_transaction_savepoint` which allows these tests to wrap
their problematic blocks in an artificial transaction which provides a
savepoint, thus preventing the full test transaction rollback derailing
the rest of the test.

zproject/backends.py

@@ -30,6 +30,7 @@ from django.contrib.auth.backends import RemoteUserBackend
 from django.contrib.staticfiles.storage import staticfiles_storage
 from django.core.exceptions import ValidationError
 from django.core.validators import validate_email
+from django.db import transaction
 from django.dispatch import Signal, receiver
 from django.http import HttpRequest, HttpResponse, HttpResponseRedirect
 from django.shortcuts import render
@@ -1208,6 +1209,38 @@ class ZulipLDAPUser(_LDAPUser):
 
         super().__init__(*args, **kwargs)
 
+    @transaction.atomic(savepoint=False)
+    def _get_or_create_user(self, force_populate: bool = False) -> UserProfile:
+        # This function is responsible for the core logic of syncing
+        # a user's data with ldap - run in both populate_user codepath
+        # and just-in-time user creation upon first login via LDAP.
+        #
+        # To ensure we don't end up with corrupted database state,
+        # we need to run these operations atomically.
+        return super()._get_or_create_user(force_populate=force_populate)
+
+    def _populate_user(self) -> None:
+        """
+        Populates our User object with information from the LDAP directory.
+        """
+        assert isinstance(self._user, UserProfile)
+        user_profile = self._user
+        original_role = user_profile.role
+
+        # _populate_user() will make whatever changes to the user's attributes
+        # that it needs - possibly changing the user's role multiple times e.g.
+        # as it cycles through various role setters in AUTH_LDAP_USER_FLAGS_BY_GROUP.
+        #
+        # For that reason, we want to only look at the final role value after
+        # it is executed. This is the actual change (if any) that should take place.
+        # This allows us to call do_change_user_role only once.
+        super()._populate_user()
+        if user_profile.role != original_role:
+            # Change the role properly, updating system groups.
+            updated_role = user_profile.role
+            user_profile.role = original_role
+            do_change_user_role(user_profile, updated_role, acting_user=None)
+
     def _get_groups(self) -> _LDAPUserGroups:
         groups = super()._get_groups()
         if settings.DEVELOPMENT: