Commit Graph

859 Commits

Author SHA1 Message Date
Lauryn Menard
ae7e105ce9 analytics: Use realm_str to generate per realm URL.
Instead of making a database query for each realm, use realm_str
to generate the URL with Realm.host_for_subdomain static method.
2023-11-01 13:08:49 -07:00
Sahil Batra
e458b73a01 user_groups: Move constants for system group names to a new class.
This commit moves constants for system group names to a new
"SystemGroups" class so that we can use these group names
in multiple classes in models.py without worrying about the
order of defining them.
2023-11-01 10:42:56 -07:00
Lauryn Menard
1112cbc822 analytics: Use activity_details_template.html for installation page. 2023-11-01 10:31:04 -07:00
Lauryn Menard
c996bb3c19 corporate: Move functions for realm installation data to new file.
Moves two functions in corporate/lib/stripe.py that are used to
get data for the main installation activity analytics page to a
separate file: corporate/lib/analytics.py.

Also, updates these functions for the possibility of realm being
None for a Customer object.
2023-10-30 10:08:45 -07:00
Lauryn Menard
b19a7a8b8a analytics: Remove Durations tab and hourly columns. 2023-10-27 12:19:13 -07:00
Alya Abbott
f8f741c4a2 analytics: Replace get_realm_day_counts with a faster query.
Use analytics_realmcount to improve query runtime. There are two
known difference with previous query:
- Messages that are deleted after hourly stats are aggregated
are not decremented in new query.
- Messages sent since the hourly aggregation last ran are not counted.
2023-10-26 12:54:56 -07:00
Mateusz Mandera
986f2fd962 send_analytics_to_remote_server: Rename to ..._to_push_bouncer.
The former name is kind of misleading - this function is for the remote
server to send analytics to the push bouncer. Under our usual
terminology, a "remote server" is a self-hosted Zulip server. So data is
sent FROM not TO a remote server.
2023-10-25 11:09:49 -07:00
Mateusz Mandera
6116679a95 remote_support: Remote redundant remote_installation_stats_link context.
This function was used in the html template in an early version of
PR #27033 but then ended up unused and didn't get cleaned up.
2023-10-17 18:05:27 -07:00
Lakshay Mittal
d94bdb3900 stats: Support fetching StreamCount based on stream_id.
As a preliminary step for including graphs of StreamCount data in our
analytics pages, add API support for fetching the chart data.

Care is taken to limit access to streams that the current user has
access to, which isn't necessary in similar views where the current
user is a server administrator by assumption.

Fixes part of #19653.

Co-authored-by: Tim Abbott <tabbott@zulip.com>
2023-10-16 15:10:14 -07:00
Anders Kaseorg
a50eb2e809 mypy: Enable new error explicit-override.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-12 12:28:41 -07:00
Anders Kaseorg
84a196641a test_support_views: Move incorrectly class-scoped code inside method.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-12 11:22:23 -07:00
Lauryn Menard
da4b54d414 analytics: Remove out of date tabs from main activity page.
Removes the various 'ad_hoc_queries' in the main activity page
that no longer have relevant data or information.
2023-10-05 12:31:57 -07:00
Lauryn Menard
40e6455579 analytics: Move integrations by client to separate activity page.
Moves the tab in the main installation activity page to be a
separate page: "/activity/integrations".
2023-10-05 12:31:57 -07:00
Lauryn Menard
0f009c439d analytics: Move remote server view to separate file.
Moves the view for "/activity/remove" from the main installation
activity view to a separate file.
2023-10-05 12:31:57 -07:00
Mateusz Mandera
f71e2c8247 support: Add basic support endpoint for remote servers. 2023-10-04 16:55:47 -07:00
Lauryn Menard
afc1d2a409 activity: Add separate page for remote server information/table.
Moves the "Remote Zulip servers" tab in the "/activity" page for
an installation to a separate page, "/activity/remote".

Prototype for moving other tabs in "/activity" to separate pages.
2023-10-04 12:48:39 -07:00
Alex Vandiver
71e297efb4 realm: Differentiate reserved realms from in-use realms.
Fixes: #23896.
2023-09-25 12:48:14 -07:00
Anders Kaseorg
2665a3ce2b python: Elide unnecessary list wrappers.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-09-13 12:41:23 -07:00
Anders Kaseorg
6988622fe8 ruff: Enable B023 Function definition does not bind loop variable.
Python’s loop scoping is misdesigned, resulting in a very common
gotcha for functions that close over loop variables [1].  The general
problem is so bad that even the Go developers plan to break
compatibility in order to fix the same design mistake in their
language [2].

Enable the Ruff rule function-uses-loop-variable (B023) [3], which
conservatively prohibits functions from binding loop variables at all.

[1] https://docs.python-guide.org/writing/gotchas/#late-binding-closures
[2] https://go.dev/s/loopvar-design
[3] https://beta.ruff.rs/docs/rules/function-uses-loop-variable/

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-09-11 18:03:45 -07:00
Alex Vandiver
b94402152d models: Always search Messages with a realm_id or id limit.
Unless there is a limit on `id`, always provide a `realm_id` limit as
well.  We also notate which index is expected to be used in each
query.
2023-09-11 15:00:37 -07:00
Alex Vandiver
4d72f81257 analytics: Only join through messages once.
Using `COUNT(*) FILTER (WHERE ...)` allows getting counts of different
subsets with only one giant join.  This makes the query significantly
more performant.
2023-09-11 15:00:37 -07:00
Anders Kaseorg
0ce6dcb905 mypy: Upgrade mypy from 1.4.1 to 1.5.1.
_default_manager is the same as objects on most of our models. But
when a model class is stored in a variable, the type system doesn’t
know which model the variable is referring to, so it can’t know that
objects even exists (Django doesn’t add it if the user added a custom
manager of a different name). django-stubs used to incorrectly assume
it exists unconditionally, but it no longer does.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-09-07 17:51:42 -07:00
Anders Kaseorg
81bd63cb46 ruff: Fix PIE808 Unnecessary start argument in range.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-09-01 14:57:01 -07:00
Anders Kaseorg
e32366638a requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-17 17:05:34 -07:00
Anders Kaseorg
710d1f7f51 analytics: Do not reseed the global random generator.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-15 17:57:16 -07:00
Steve Howell
751b8b5bb5 tests: Flush per-request caches automatically for query counts. 2023-08-11 11:09:34 -07:00
Anders Kaseorg
ca40e60469 ruff: Enable PERF rules.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-07 17:23:55 -07:00
Anders Kaseorg
562a79ab76 ruff: Fix PERF401 Use a list comprehension to create a transformed list.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-07 17:23:55 -07:00
Anders Kaseorg
c2c96eb0cf python: Annotate type aliases with TypeAlias.
This is not strictly necessary but it’s clearer and improves mypy’s
error messages.

https://docs.python.org/3/library/typing.html#typing.TypeAlias
https://mypy.readthedocs.io/en/stable/kinds_of_types.html#type-aliases

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-07 10:02:49 -07:00
Anders Kaseorg
2ae285af7c ruff: Fix PLR1714 Consider merging multiple comparisons.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-07-23 15:21:33 -07:00
Anders Kaseorg
3b09197fdf ruff: Fix RUF015 Prefer next(...) over single element slice.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-07-23 15:20:53 -07:00
Anders Kaseorg
143baa4243 python: Convert translated positional {} fields to {named} fields.
Translators benefit from the extra information in the field names, and
need the reordering freedom that isn’t available with multiple
positional fields.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-07-18 15:19:07 -07:00
Anders Kaseorg
7e707270f0 models: Convert deprecated index_together option to indexes.
index_together is slated for removal in Django 5.1:
https://docs.djangoproject.com/en/4.2/internals/deprecation/#deprecation-removed-in-5-1

We set the optional index names to match the previously generated
index names to avoid adding new migrations.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-07-12 07:12:43 -07:00
Alex Vandiver
6c3969f893 name_restrictions: Reject anything with zulip or kandra in it.
This is primarily to prevent impersonation, such as `zulipteam`.  We
only enable these protections for CORPORATE_ENABLED, since `zulip` is
a reasonable test name for self-hosters.
2023-06-23 10:45:40 -07:00
Lauryn Menard
47d0bb6b7d analytics: Replace "private message" with "direct message". 2023-06-19 13:40:08 -07:00
Mateusz Mandera
2b7877bcb4 support: Make user search by email case-insensitive.
It's pretty troublesome to fail to find a user in our database because
they used a different capitalization when signing up.
2023-05-16 12:38:12 -07:00
Mateusz Mandera
632e856240 support: Add option to delete user from /activity/support. 2023-05-16 12:38:12 -07:00
Alex Vandiver
505eec4bac analytics: Create a RealmAuditLog entry when subscribing test users. 2023-05-15 16:09:44 -07:00
Mateusz Mandera
24d9d3d90c support: Add option to upgrade an org to Plus plan on /activity/support.
This is so that we don't need ops to run the
manage.py switch_realm_from_standard_to_plus_plan command on the server
to handle these upgrades.
2023-04-10 17:32:45 -07:00
Mateusz Mandera
1a15449e16 support: Rename "downgrade plan" to "modify plan".
Just a naming change to facilitate adding more options to that <select>
- mainly, an option to upgrade the plan to Plus.
2023-04-10 17:32:45 -07:00
Sahil Batra
f684d36710 test_classes: Add submit_realm_creation_form helper.
This commit adds a new helper submit_realm_creation_form,
similar to existing submit_reg_form_for_user, to avoid
duplicate code for creating realms in tests.
2023-03-27 15:44:42 -07:00
Sahil Batra
54771cfe94 registration: Create PreregistrationRealm object when creating realm.
This commit adds code to create PreregistrationRealm object when
creating realm and set it to the content_object field of
Confirmation object.
2023-03-27 15:44:42 -07:00
Alex Vandiver
0b80397cfd web: Save a 301 redirect from /support to /support/. 2023-03-24 14:51:01 -07:00
Anders Kaseorg
afa218fa2a semgrep: Detect some unsafe uses of markupsafe.Markup.
Use the built-in HTML escaping of Markup("…{var}…").format(), in order
to allow Semgrep to detect mistakes like Markup("…{var}…".format())
and Markup(f"…{var}…").

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-22 11:23:27 -07:00
Alex Vandiver
330141f55d invites: Switch new LIMITED-plan heuristic to enforcing. 2023-03-16 11:41:49 -07:00
Anders Kaseorg
2d9b2a2a05 models: Remove type prefixes from __str__ values.
The Django convention is for __repr__ to include the type and __str__
to omit it.  In fact its default __repr__ implementation for models
automatically adds a type prefix to __str__, which has resulted in the
type being duplicated:

    >>> UserProfile.objects.first()
    <UserProfile: <UserProfile: emailgateway@zulip.com <Realm: zulipinternal 1>>>

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-08 22:56:55 -08:00
Alex Vandiver
36da7783ce invites: _max_invites is currently never None.
dc1eeef30a made the column nullable, with the meaning for null of
"use the current `settings.INVITES_DEFAULT_REALM_DAILY_MAX`."
However, 8a95526ced switched to calling `do_change_plan_type` during
realm creation, which sets `realm.max_invites` based on the plan type,
thus ensuring that no new realms have their `_max_invites` set to
null.

Check `max_invites` instead of `_max_invites`.  This requires test
adjustments for the fact that `apply_invite_realm_heuristics` is now
run.
2023-03-07 15:04:39 -08:00
Anders Kaseorg
d3efd4c095 python: Import F, Q, QuerySet from their canonical module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-05 14:46:28 -08:00
Anders Kaseorg
0628c3cac8 migrations: Import BaseDatabaseSchemaEditor from its canonical module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-05 14:46:28 -08:00
Alex Vandiver
04e7621668 upload: Rename upload_message_image_from_request.
The table is named Attachment, and not all of them are images.
2023-03-02 16:36:19 -08:00