791 Commits

Author SHA1 Message Date
Anders Kaseorg
d88969e865 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit fd55da4ed8)
2024-07-27 09:27:23 -07:00
Anders Kaseorg
8ce54599c7 ruff: Fix PERF403 Use a dictionary comprehension instead of a for-loop.
This is a preview rule, not yet enabled by default.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 87992b8b29)
2024-03-07 15:07:12 -08:00
Anders Kaseorg
9ea290d5ae python: Reformat with Ruff formatter.
https://docs.astral.sh/ruff/formatter/

Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 570f3dd447)
2024-03-07 15:07:12 -08:00
Mateusz Mandera
96001b19fc update_analytics_count: Use a correct lock mechanism.
Adds a re-usable lockfile_nonblocking helper to context_managers.

Relying on naive `os.mkdir` is not enough especially now that the
successful operation of this command is necessary for push notifications
to work for many servers.

We can't use `lockfile` context manager from
`zerver.lib.context_managers`, because we want the custom behavior of
failing if the lock can't be acquired, instead of waiting.
That's because if an instance of this gets stuck, we don't want to start
queueing up more processes waiting forever whenever the cronjob runs
again and fail->exit is preferrable instead.

(cherry picked from commit f61ed58c8f)
2024-03-07 15:01:15 -08:00
Alex Vandiver
aee36b2b49 tests: Clear in-memory Client caches before testing query counts.
This makes counts more apples-to-apples comparable when run
back-to-back.

(cherry picked from commit a84de411a9)
2024-03-07 09:48:20 -08:00
Lauryn Menard
85bd8dd33d push-bouncer: Exclude LoggingCountStats with partial data.
LoggingCountStats with a daily duration and that are directly stored
on the RealmCount table (not via aggregation in process_count_stat),
can be in a state, after the hourly cron job to update analytics
counts, where the logged value will be live-updated later, because
the end time for the stat is still in the future.

As these logging counts are designed to be used on the self-hosted
installation for either debugging or rate limiting, sending these
partial/incomplete counts to the bouncer has low value.

(cherry picked from commit 40daf10a74)
2024-03-01 09:26:25 -08:00
Anders Kaseorg
2e4785f51c requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 93198a19ed)
2024-02-16 12:17:34 -08:00
Anders Kaseorg
d07d773809 models: Extract zerver.models.scheduled_jobs.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-05 10:32:54 -05:00
Anders Kaseorg
26811c5049 models: Extract zerver.models.clients.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-05 10:32:54 -05:00
Anders Kaseorg
086df4a81e models: Extract zerver.models.realms.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-05 10:32:54 -05:00
Anders Kaseorg
a8b5a99281 models: Extract zerver.models.groups.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-05 10:32:54 -05:00
Anders Kaseorg
ee85ac5433 models: Extract zerver.models.users.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-05 10:32:54 -05:00
Sahil Batra
03323b0124 push_notifications: Enforce max user count on self managed plan.
We do not support sending push notifications for realms having
more than 10 users on self managed plan.
2023-12-15 11:03:42 -08:00
Aman Agrawal
34a1be80a4 model: Use same name for legacy plan across the app. 2023-12-14 08:22:25 -08:00
Tim Abbott
6308e07e53 billing: Standardize remote server plan type IDs.
This will likely save us at least one headache.
2023-12-13 16:40:44 -08:00
Lauryn Menard
a897d68d93 support: Add active plan information to remote server activity.
Adds three columns to the remote server activity chart and updates
the chart key for the third of those columns.

The first is the plan name. If there are multiple plans with a
status under the live threshhold, then we send "See support view".

The second is the plan status. If there are multiple plans, then
we send "Multiple plans".

The third is the estimated annual revenue for the plan. Note that
for free trials, this will be calculated as if the plan was paid
for 12 months (so a full year).

If there is no plan for the server under the live threshold or at
all then "---" is inserted into the table row. Note that 100%
sponsored servers/realms would fall into this category.
2023-12-13 13:43:00 -08:00
Lauryn Menard
68d117e60a support: Add sponsorship request information to remote support view. 2023-12-13 10:28:11 -08:00
Lauryn Menard
e8500fbdb0 support: Include sponsorship request for remote server support view.
Preparation for updating the sponsorship forms template to include
information about the latest sponsorship request if sponsorship is
pending.
2023-12-13 10:28:11 -08:00
Alex Vandiver
19575b46f7 remote_activity: Hide deactivated remote servers. 2023-12-12 20:36:03 -08:00
Alex Vandiver
6d7037e867 analytics: Show latest number in "Analytics users", not max. 2023-12-12 12:09:51 -08:00
Lauryn Menard
449714245e support: Exclude system bot realm from remote realm query. 2023-12-12 08:04:02 -08:00
Prakhar Pratyush
c1daabd3c0 remote_server: Rename to 'send_server_data_to_push_bouncer'.
This commit renames 'send_analytics_to_push_bouncer'
to 'send_server_data_to_push_bouncer'.
2023-12-11 14:07:39 -08:00
Prakhar Pratyush
d763fae9d0 remote_server: Eliminate separate realms-only code path.
Given that most of the use cases for realms-only code path would
really like to upload audit logs too, and the others would likely
produce a better user experience if they upoaded audit logs, we
should just have a single main code path here i.e.
'send_analytics_to_push_bouncer'.

We still only upload usage statistics according to documented
option, and only from the analytics cron job.

The error handling takes place in 'send_analytics_to_push_bouncer'
itself.
2023-12-11 14:07:39 -08:00
Lauryn Menard
7186f75b58 support: Add sponsorship/discount support forms to remote realms.
If there are remote realms attached to the remote server, adds
the forms to be able to update sponsorship requests, approve
sponsorship and update a discount.
2023-12-11 10:33:50 -08:00
Lauryn Menard
c2a76ccb3e support: Clean up template variables for remote server support. 2023-12-11 10:33:50 -08:00
Lauryn Menard
d079a13760 support: Use shared template for current plan details on support views.
Moves the section in support views for any current plan details
to a new template: `templates/analytics/current_plan_details.html`.

Also, updates the PlanData dataclass to have a boolean that checks
if the current plan tier is the self-hosted legacy plan.
2023-12-06 11:01:26 -08:00
Anders Kaseorg
223b626256 python: Use urlsplit instead of urlparse.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-05 13:03:07 -08:00
Anders Kaseorg
3853fa875a python: Consistently use from…import for urllib.parse.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-05 13:03:07 -08:00
Lauryn Menard
ba3279e959 support: Add remote realm stub to remote server information.
Adds a small section for any remote realms attached to the remote
server in the support view.
2023-12-05 12:48:42 -08:00
Lauryn Menard
cfd61669e0 support: Add plan type to remote server information.
Updates `get_plan_type_string` for RemoteZulipServer plan types and
capitalizes the strings used for Realm plan types.

Also changes the string for Realm.PLAN_TYPE_STANDARD_FREE to be
"Standard free" instead of "open source" as that is used for any
100% sponsored organization, which is not restricted to open-source
projects.
2023-12-05 12:48:42 -08:00
Lauryn Menard
6c5b419267 support: Add downgrade current plan actions to remote servers. 2023-12-05 12:48:42 -08:00
Lauryn Menard
f5ab9419d2 support: Update billing modality for remote server current plan. 2023-12-05 12:48:42 -08:00
Anders Kaseorg
8a7916f21a python: Consistently use from…import for datetime.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-05 12:01:18 -08:00
Tim Abbott
95b39bc8bf analytics: More fixes to zilencer urls import logic. 2023-12-05 10:52:00 -08:00
Tim Abbott
28d736d63b analytics: Don't import zilencer-specific code unconditionally.
This fixes the production server not working when zilencer is not
enabled.
2023-12-05 10:36:35 -08:00
Prakhar Pratyush
14b1539647 remote_activity: Add 'guest users' and 'non guest users' count column.
This commit adds two columns named 'Guest users' and
'Non guest users' to respresent count of such users.

We query 'RemoteRealmAuditLog' to get the data.
2023-12-05 10:04:41 -08:00
Lauryn Menard
dc0b8bcb6b analytics: Update query for remote_push_devices in remote activity.
Updates query to count the distinct pairs of user_id and user_uuid
on the RemotePushDeviceToken table.
2023-12-04 11:50:18 -08:00
Lauryn Menard
69f6d3dcb1 support: Approve sponsorship for remote server customer.
Adds ability to approve a sponsorship request for a customer
attached to a remote server via the remote server support view.
2023-12-03 14:27:07 -08:00
Lauryn Menard
2218e667a6 support: Update remote server customer sponsorship pending status.
Adds ability to update the sponsorship pending status of a customer
attached to a remote server via the remote server support view.
2023-12-03 14:27:07 -08:00
Lauryn Menard
4250e9c1c9 support: Update remote server customer default discount.
Adds ability to update the default discount for a customer attached
to a remote server via the remote server support view.
2023-12-03 14:27:07 -08:00
Lauryn Menard
38eaa4b958 support: Add plan data to remote server support view context. 2023-12-03 14:27:07 -08:00
Lauryn Menard
1d685bd198 support: Add success message to remote servers support view context. 2023-12-03 14:27:07 -08:00
Lauryn Menard
6d66248d3d support: Get plan data via BillingSession for support views. 2023-12-03 14:27:07 -08:00
Lauryn Menard
8d992405a6 analytics: Get customer discount for support views. 2023-12-03 14:27:07 -08:00
Lauryn Menard
5135acd9e3 support: Use process_support_view_request for plan modifications.
Updates the support view to use process_support_view_request to
process upgrade or downgrade modifications currently implemented
for active plans.
2023-12-03 14:27:07 -08:00
Lauryn Menard
4fb564026d corporate: Move void_all_open_invoices to BillingSession. 2023-12-03 14:27:07 -08:00
Lauryn Menard
97d33a4363 support: Update billing modality via process_support_view_request. 2023-12-03 14:27:07 -08:00
Lauryn Menard
5d25cab42b analytics: Create process_support_view_request BillingSession method.
Creates a process_support_view_request method for BillingSession
to process the various support requests that relate to the billing
system.

Moves approve_realm_sponsorship, update_realm_sponsorship_status,
and attach_discount_to_realm to this new BillingSession method.

Adds a new abstract property to BillingSession to have a string
value, billing_entity_display_name, to use for support messages
sent when these requests are processed.
2023-12-03 14:27:07 -08:00
Lauryn Menard
0679bc044a corporate: Make references to billing modality consistent.
The "send_invoice" and "charge_automatically" strings used by stripe
for the `collection_method` are referred to both as the "billing
method" and "billing modality" in the billing code.

Because we send this as data to stripe as either `collection_method`
or `billing_modality`, renames any references that are any form of
"billing method".
2023-12-03 14:27:07 -08:00
Mateusz Mandera
c9b0602320 tests: Create default RemoteRealms in populate_db.
This default setup will be more realistic, matching the ordinary
conditions for a modern server.
Especially needed as we add bouncer code that will expect to have
RemoteRealm entries for realm_uuid values for which it receives
requests.
2023-12-03 09:51:45 -08:00