APNs tokens are provided by the client in hex, and we store them in
hex. The existing code which attempts to "validate" them by parsing
them as base64 only works because base64 is a superset of hex.
Enforce that APNs tokens are hex, and remove all of the pieces of test
code which were incorrectly passing them in as base64 strings.
In the analytics tests, where users are created as part of the set up
of the test, we want to get users by the delivery email and not the
visible-to-other-users email field.
Prep for setting default new user email visibility in realm creation
by the organization type.
Updates the labels in the"Messages set by client" chart so that
the Flutter app is no longer labeled as "beta".
The React Native app is now labeled as "old". And the older
Android and iOS apps are now labeled as "ancient".
If this is called on a user without is_mirror_dummy=True, that seems
certain to be a bug. Therefore, an assert is preferable in order to
catch this, rather than returning early with noop like some other
function such as do_deactivate_user.
We were using admins group as a hardcoded value for the default of
`can_remove_subscribers_group`, now we use a function to get the value
of the default group.
This commit adds a new `group_size` field to the `DirectMessageGroup`
model, and backfills its value to each of the existing direct message
groups.
Fixes part of #25713
Instead of the PUSH_NOTIFICATIONS_BOUNCER_URL and
SUBMIT_USAGE_STATISTICS settings, we want servers to configure
individual ZULIP_SERVICE_* settings, while maintaining backward
compatibility with the old settings. Thus, if all the new
ZULIP_SERVICE_* are at their default False value, but the legacy
settings are activated, they need to be translated in computed_settings
to the modern way.
This commit renames the "Huddle" Django model class to
"DirectMessageGroup", while maintaining the same table --
"zerver_huddle".
Fixes part of #28640.
With `realm_active_humans` no longer dependent on the per-user rows,
there is no reason to preserve them -- any measure of "was a user
active" should look directly at the much richer RealmAuditLog. This
removes the bulk of the UserCount table, since the remaining rows all
require user interaction of some sort to produce rows.
This makes it no longer dependent on active_users_audit:is_bot:day,
which subsequent commits will make a RealmCount, not UserCount, query.
This folds the same behaviour of `active_users_audit` directly into
the query; however, only running over active users, using the index
from the earlier commit, and using the new `DISTINCT ON` formulation
make this a fast query compared to `active_users_audit:is_bot:day` +
the old `realm_active_humans::day`.
Updates the labels in the "Messages sent by client" analytics chart
for the user-agent/client names for the Flutter mobile app, which
can be "ZulipFlutter" or "ZulipMobile/flutter".
Fixes#28220.
This commit adds include_realm_default_subscriptions parameter
to the invite endpoints and the corresponding field in
PreregistrationUser and MultiuseInvite objects. This field will
be used to subscribe the new users to the default streams at the
time of account creation and not to the streams that were default
when sending the invite.
Fixes#29632.
The issue description explains this well:
We currently recalculate `currently_used_upload_space_bytes` every file
upload, by dint of calling `flush_used_upload_space_cache` on
save/delete, and then immediately calling
`user_profile.realm.currently_used_upload_space_bytes()` in
`notify_attachment_update`. Since this walks the Attachments table,
recalculating this can take seconds in large realms.
Switch this to using a CountStat, so we don't need to walk significant
chunks of the Attachment table when we upload an attachment. This will
also give us a historical daily graph of usage.
The previous implementation using Django's `get_or_create` for
`do_increment_logging_stat` involved two separate database queries,
potentially leading to race conditions.
Use an `ON CONFLICT ... DO UPDATE` (aka "upsert") query, which
eliminates race conditions and improves performance. This is mildly
complicated due to the different unique indexes across the various
tables, and the need for bug-for-bug compatibility with the previous
implementation.
Fixes#28947.
Co-authored-by: Alex Vandiver <alexmv@zulip.com>
The "invites" worker exists to do two things -- make a Confirmation
object, and send the outgoing email. Making the Confirmation object
in a background process from where the PreregistrationUser is created
temporarily leaves the PreregistrationUser in invalid state, and
results in 500's, and the user not immediately seeing the sent
invitation. That the "invites" worker also wants to create the
Confirmation object means that "resending" an invite invalidates the
URL in the previous email, which can be confusing to the user.
Moving the Confirmation creation to the same transaction solves both
of these issues, and leaves the "invites" worker with nothing to do
but send the email; as such, we remove it entirely, and use the
existing "email_senders" worker to send the invites. The volume of
invites is small enough that this will not affect other uses of that
worker.
Fixes: #21306Fixes: #24275
This prevents users from hammering the invitation endpoint, causing
races, and inviting more users than they should otherwise be allowed
to.
Doing this requires that we not raise InvitationError when we have
partially succeeded; that behaviour is left to the one callsite of
do_invite_users.
Reported by Lakshit Agarwal (@chiekosec).
Updates translated JsonableError strings that relate to streams
to use channel instead of stream. Separated from other error string
updates as this is a dense area of changes for this rename.
Part of stream to channel rename project.
Updates the labels for the "Messages sent by recipient type" chart
to use "channel", and updates the error message that would be sent
for the "messages_sent_by_stream" chart (that has not yet been
implemented) for a missing channel/stream.
Part of the stream to channel rename project.
Replaced HUDDLE attribute with DIRECT_MESSAGE_GROUP using VS Code search,
part of a general renaming of the object class.
Fixes part of #28640.
Co-authored-by: JohnLu2004 <JohnLu10212004@gmail.com>
The previous logic incorrectly used the server-level number of users
even when a (presumably smaller) realm-level count was available.
Fixes a bug introduced in 2e1ed4431a.
View functions in `analytics/views/support.py` are moved to
`corporate/views/support.py`.
Shared activity functions in `analytics/views/activity_common.py`
are moved to `corporate/lib/activity.py`, which was also renamed
from `corporate/lib/analytics.py`.
To estimate the annual recurring revenue for remote server and
remote realm CustomerPlans, we prefetch the current license
ledger as part of the CustomerPlan query.
Also adds a select related to the remote realm audit log query
so that we don't go to the database for the remote realm ID.
With the test added in the previous commit, the query count for
the remote activity view goes from 27 to 11, as we are no longer
hitting the database multiple times for every current plan or
for every remote realm with audit log data.
Refactors get_customer_plan_renewal_amount so that a license
ledger is always passed and make_end_of_cycle_updates_if_needed
does not need to be called.
Creates multiple remote servers and remote realms with active
plan data and audit logs.
Shows multiple small queries to get revenue data from plan license
ledger, as well as remote server and remote realm IDs.
Instead of showing the next invoice date for the plan, show the
date for the next billing cycle start (e.g. the next plan renewal
charge date), except for plans currently on a free trial.
For plans on a free trial, the next plan renewal date will be when
the free trial ends, which is stored as the next invoice date on
the plan.
Instead of querying the database for every remote server and realm
in the remote activity chart, we now get the server and realm data
for the installation in two queries.
Adds columns for remote realm ID, name and organization type. If
a remote server has remote realms attached that are not marked
as deactivated by the remote server, then there will be a row in
the chart for each remote realm (which duplicates some remote
server data).
Updates the plan data, revenue and user counts to be for the realm
if present and otherwise for the server.
Updates the user counts to be total users and guest users, instead
of non guest and guest users.
The total row for mobile data (users and pushes forwarded) sums
each remote server's data once, so while the column duplicates
data, the total row should be an accurate total for the installation.
Adds 5 queries to the remote activity page test. One is for the
additional query for the remote realm plans. The other four are
getting the remote realm object and then the user count data for
the two remote realms in the test.
Merges the two charts remaining to have just one chart for the
realm activity view.
Removes the columns for different clients and adds a column to
show/sort by the user type (human or bot type).
Deletes templates/analytics/activity.html because it is no
longer used for any activity pages/views.
