Commit Graph

187 Commits

Author SHA1 Message Date
Lauryn Menard
8561800676 video-calls: Add Zoom Serverto Server OAuth integration.
Adds a second Zoom integration that uses the Zoom Server to Server
OAuth app process. Only one of the two Zoom integrations can be
configured on a Zulip server.

Adds a cache for the access token from the Zoom server so that it
can be used by the server to create meetings for the approximate
duration of the access token

In the web-app compose box, if the user's delivery email does not
match a user on the configured Zoom account for the server to server
integration, then a compose box error banner will be shown when the
error response is received after clicking/selecting the video or
audio call button.

Also updates the production documentation for the both types of Zoom
integration apps (Server to Server and General). The General app
process for Zoom now requires unlisted apps to go through their
review process, which we now have documented.

Fixes #33117.
2025-02-13 16:35:43 -08:00
Aman Agrawal
2f5aea6604 message_summary: Add setting to put a monthly rate limit per user. 2025-02-06 12:03:24 -08:00
Aman Agrawal
7fb767c080 message_summary: Move cost per token configuration to settings. 2025-02-06 12:03:24 -08:00
Aman Agrawal
099bab986f settings: Remove unused setting.
Since TOPIC_SUMMARIZATION_MODEL is already defined in
dev_settings to a groq model, this is not used.
2025-02-06 12:03:24 -08:00
Simon Michalke
8de5de66fa avatar: Add option to disable gravatars per realm.
The functionality of gravatar can break anonymity if
the user has had a gravatar account set up previously.

This option allows specifically cloud instances to
have gravatar disabled selectively.
2025-01-30 18:46:02 -08:00
Mateusz Mandera
7390eb2ed0 zilencer: Rename registration takeover to registration transfer.
This is the final naming that we want, compared to the naming we merged
in #32399.
Includes renaming the API endpoints, but that should be fine as the
original PR was just merged and this isn't deployed anywhere.
2025-01-30 14:32:36 -08:00
Alex Vandiver
4dfb285b5b message_summary: Reorganize settings and secrets.
This allows arbitrary litellm-supported backends.
2025-01-29 14:55:35 -08:00
Mateusz Mandera
4e22a79e6a zilencer: Add flow for a server to reclaim its registration.
If the server controls the registration's hostname, it can reclaim its
registration credentials. This is useful, because self-hosted admins
frequently lose the credentials when moving their Zulip server to a
different machine / deployment method.

The flow is the following:
1. The host sends a POST request to
   /api/v1/remotes/server/register/takeover.
2. The bouncer responds with a signed token.
3. The host prepares to serve this token at /api/v1/zulip-services/verify and
   sends a POST to /remotes/server/register/verify_challenge endpoint of
   the bouncer.
4. Upon receiving the POST request, the bouncer GETS
   https://{hostname}/api/v1/zulip-services/verify, verifies the secret and
   responds to the original POST with the registration credentials.
5. The host can now save these credentials to it zulip-secrets.conf file
   and thus regains its push notifications registration.

Includes a global rate limit on the usage of the /verify_challenge
endpoint, as it causes us to make outgoing requests.
2025-01-28 11:10:50 -08:00
Aman Agrawal
9f71f4578b portico: Replace "Find accounts" link with "Log in".
Fixes #32199

We only need a log in button since that will take users to
"/accounts/go" if we are on a non-realm specific URL.

"/accounts/go" already has link to go to "Find accounts" page.
2025-01-24 14:55:47 -08:00
Maneesh Shukla
4362c8d3c9 password: Add password_max_length to register response.
This allows clients to render UI that disallows invalid passwords
before sending them to the server.
2025-01-13 11:47:34 -08:00
opmkumar
5b0c55fda3 realm: Add option to schedule data deletion while deactivating.
Introduce a feature to schedule realm data deletion time during realm
deactivation. This includes a server-level setting to configure the
minimum and maximum allowed deletion days.

Co-authored-by: Ujjawal Modi <umodi2003@gmail.com>
Co-authored-by: Lauryn Menard <lauryn@zulip.com>

Fixes #24677.
2024-12-18 23:06:12 -08:00
Aman Agrawal
35ffaff1f6 tusd: Allow user to upload files during preregistration.
Allow user to upload files during preregistration. This will be used
to upload import data.

Co-authored-by: Alex Vandiver <alexmv@zulip.com>
2024-12-02 10:21:45 -08:00
Tim Abbott
30c54d023b settings: Write fewer values into template settings files.
This makes it easier for us to change the default values for these
settings, and also just helps keep settings files clear of values that
the relevant organization didn't choose themselves.

There's about a dozen more settings that could receive similar
treatment; I just went with the easiest ones.
2024-11-06 16:43:44 -08:00
Tim Abbott
f54817d78d groups: Enable group-valued settings in production. 2024-10-31 12:04:51 -07:00
Alex Vandiver
e125ad823d exports: Add a separate bucket for realm exports.
This allows finer-grained access control and auditing.  The links
generated also expire after one week, and the suggested configuration
is that the underlying data does as well.

Co-authored-by: Prakhar Pratyush <prakhar@zulip.com>
2024-09-20 15:43:49 -07:00
Alex Vandiver
24d110f063 settings: Increase default max file upload size to 100MB.
This also _lowers_ the default nginx client_max_body_size, since that
no longer caps the upload file size.
2024-09-19 11:37:29 -07:00
Mateusz Mandera
a36f906d1a presence: Add history_limit_days param to the API.
This param allows clients to specify how much presence history they want
to fetch. Previously, the server always returned 14 days of history.
With the recent migration of the presence API to the much more efficient
system relying on incremental fetches via the last_update_id param added
in #29999, we can now afford to provide much more history to clients
that request it - as all that historical data will only be fetched once.

There are three endpoints involved:
- `/register` - this is the main useful endpoint for this, used by API
clients to fetch initial data and register an events queue. Clients can
pass the `presence_history_limit_days` param here.
- `/users/me/presence` - this endpoint is currently used by clients to
update their presence status and fetch incremental data, making the new
functionality not particularly useful here. However, we still add the
new `history_limit_days` param here, in case in the future clients
transition to using this also for the initial presence data fetch.
- `/` - used when opening the webapp. Naturally, params aren't passed
here, so the server just assumes a value from
`settings.PRESENCE_HISTORY_LIMIT_DAYS_FOR_WEB_APP` and returns
information about this default value in page_params.
2024-09-10 13:15:35 -07:00
Mateusz Mandera
833dce8a13 saml: Add support for syncing user role.
Replace the SOCIAL_AUTH_SYNC_CUSTOM_ATTRS_DICT with
SOCIAL_AUTH_SYNC_ATTRS_DICT, designed to support also regular user attrs
like role or full name (in the future).

Custom attributes can stay configured as they were and will get merged
into SOCIAL_AUTH_SYNC_ATTRS_DICT in computed_settings, or can be
specified in SOCIAL_AUTH_SYNC_ATTRS_DICT directly with "custom__"
prefix.

The role sync is plumbed through to user creation, so users can
immediately be created with their intended role as provided by the IdP
when they're creating their account, even when doing this flow without
an invitiation.
2024-08-20 11:53:24 -07:00
Alex Vandiver
b42863be4b markdown: Show thumbnails for uploaded images.
Fixes: #16210.
2024-07-21 18:41:59 -07:00
Anders Kaseorg
7b8fb017c3 settings: Disable KaTeX server in development.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-17 17:58:32 -07:00
Mateusz Mandera
4a93149435 settings: Rework how push notifications service is configured.
Instead of the PUSH_NOTIFICATIONS_BOUNCER_URL and
SUBMIT_USAGE_STATISTICS settings, we want servers to configure
individual ZULIP_SERVICE_* settings, while maintaining backward
compatibility with the old settings. Thus, if all the new
ZULIP_SERVICE_* are at their default False value, but the legacy
settings are activated, they need to be translated in computed_settings
to the modern way.
2024-07-17 17:14:06 -07:00
Anders Kaseorg
0fa5e7f629 ruff: Fix UP035 Import from collections.abc, typing instead.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg
531b34cb4c ruff: Fix UP007 Use X | Y for type annotations.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg
e08a24e47f ruff: Fix UP006 Use list instead of List for type annotation.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Sahil Batra
ffb7744974 settings: Allow settings to be set to system groups only in production.
We use the already existing server level setting to only allow
settings to be set to system groups, not a named user defined
group as well, in production. But we allow to settings to be set
to any named or anonymous user group in tests and development server.

"can_mention_group" setting can be set to user defined groups
because some of the realms already do that in production.

The existing server level setting is also renamed to make it clear
that both user defined groups and anonymous groups are not allowed
if that setting is set to False.

This commit also changes the error message to be consistent for the
case when a setting cannot be set to user defined groups as per
server level and setting and when a particular setting cannot be set
to user defined groups due to the configuration of that particular
setting. For this we add a new class SystemGroupRequiredError in
exceptions.py so that we need not re-write the error message in
multiple places.
2024-07-12 09:51:49 -07:00
Alex Vandiver
5cd10ce51d s3: Allow setting a CloudFront URL prefix for avatar and emoji images. 2024-06-26 16:43:11 -07:00
Mateusz Mandera
4917e01ffb push_notifications: Migrate to FCM HTTP v1 API.
The legacy API we use via python-gcm is deprecated and about to be
disabled.

Fixes #29768.
2024-06-17 18:26:26 -07:00
Gaurav Pandey
1c5007461a topic: Add resolve topic undo grace period.
Currently we send a notification to the topic if it has been resolved
or unresolved even if there is an immediate event of resolving and
then unresolving or vice-versa. This adds a setting of
RESOLVE_TOPIC_UNDO_GRACE_PERIOD_SECONDS under which if a topic has
been unresolved after being resolved immediately and the last message
was the notification of resolving, then delete the last message and
don't send a new notification and vice-versa.

We use the new message.type field to precisely identify relevant
messages.

Fixes #19181.

Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
2024-06-10 13:34:23 -07:00
Sahil Batra
d3ea6520dc user_groups: Add server level setting disallow anonymous groups for settings.
This commit adds a server level setting which controls whether the setting
can be set to anonymous user groups. We only allow it in the tests for
now because the UI can only handle named user groups.
2024-06-03 09:45:26 -07:00
Alex Vandiver
5814583694 rabbitmq: Support non-/ vhosts.
Fixes: #30255.
2024-05-31 15:41:19 -07:00
Alex Vandiver
f246b82f67 puppet: Factor out pattern of writing a nagios state file atomically. 2024-05-24 11:31:25 -07:00
Alex Vandiver
5c2fd1de5a docs: Update Django links to our current version. 2024-05-24 10:18:37 -07:00
Prakhar Pratyush
bf2360bcf2 onboarding_steps: Remove hotspot as an onboarding_step.
Earlier, hotspots and one-time notices were the valid
type of onboarding step.

Now, one-time notice is the only valid type.

Fixes #29296.
2024-05-10 12:30:22 -07:00
Alex Vandiver
16a53d08fd default_settings: Set NAGIOS_BOT_HOST to most likely correct hostname.
Using only EXTERNAL_HOST results in:
```
CRITICAL: Error subscribing to Zulips: Account is not associated with this subdomain
```

Since system bots like the nagios bot are created in
`SYSTEM_BOT_REALM`, default to that subdomain of `EXTERNAL_HOST`.
2024-05-08 15:51:20 -07:00
Alex Vandiver
4dd801eb71 settings: Fix a typo in a comment. 2024-04-30 12:37:02 -07:00
Prakhar Pratyush
c727b36e9c soft_reactivate: Soft reactivate if group mention has < 12 members.
Earlier, we didn't soft-reactivate users for group mentions
at all because it wasn't easy to calculate group size.

Now, we will soft reactivate if the user group mentions has
less than 12 members.

We don't reactivate all users because a user group can have a
very large size, which can lead to large backlogs in the
deferred-work queue.

Fixes part of #27586.
2024-04-24 17:40:37 -07:00
Alex Vandiver
62cf6ab35c sentry: Support setting project_dsn in zulip.conf. 2024-04-24 14:40:28 -07:00
Anders Kaseorg
166c233b2a default_settings: Weaken CUSTOM_AUTHENTICATION_WRAPPER_FUNCTION type.
Fixes an import cycle that breaks mypy inference with django-stubs:

zproject.settings → zproject.configured_settings →
zproject.default_settings → zerver.models.users → django.conf →
zproject.settings

(The Django settings system is really not an appropriate place to put
functions.)

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-04-17 16:49:03 -07:00
Mateusz Mandera
066de96a86 uploads: Implement 5GB/user quota for paid orgs on Zulip Cloud.
Fixes #28621

Till now, this was actually a flat 50GB despite what the /plans/ page
says and was adjusted flexibly when somebody asked for a higher limit.

This actually implements the advertised formula, but changing it to
5GB/user since that's a more reasonable limit.

Keeps the 50GB limit for sponsored Standard Free organizations and also
places it as the floor for the quota for paid orgs, to not lower this
for tiny orgs with less than 5 users.
2024-04-15 15:08:56 -07:00
Alex Vandiver
9451d08bb9 worker: Split out worker sampling rate, and add Sentry transactions. 2024-03-21 12:46:13 -07:00
Alex Vandiver
f64b9475c1 sentry: Optionally enable tracing, with transactions and spans. 2024-03-21 12:46:13 -07:00
Alex Vandiver
b23d90ed62 puppet: Rename puppet/zulip_ops to puppet/kandra.
This makes for easier tab-completion, and also is a bit more explicit
about the expected consumer.
2024-02-06 17:56:27 -08:00
Mateusz Mandera
649b4885e8 remote_billing: Add rate-limiting for confirmation email sending.
These should be rate-limited by both IP using our regular
sends_email_by_ip bucket as well as by server, using a new bucket
dedicated to this.
2024-01-18 12:06:16 -08:00
Mateusz Mandera
80f5963bbc auth: Add a configurable wrapper around authenticate calls. 2024-01-15 12:18:48 -08:00
Aman Agrawal
f96d58cf02 settings: Set default free trial days for basic plan to 30 days. 2024-01-12 08:50:51 -08:00
Alex Vandiver
75d6f35069 s3: Add a setting for S3 addressing style.
This controls if boto3 attempts to use
`https://bucketname.endpointname/` or `https://endpointname/bucket/`
as its prefix.  See
https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html

Fixes: #28424.
2024-01-05 11:12:18 -08:00
Aman Agrawal
ce56e19d1c stripe: Separate activation of free trial for remote realm/server.
Add a separate setting to only enable free trial for remote
realm / server.
2023-12-10 15:18:01 -08:00
Aman Agrawal
ccd60bc7e2 settings: Use original FREE_TRIAL_DAYS for cloud free trials.
* Renamed FREE_TRIAL_DAYS to CLOUD_FREE_TRIAL_DAYS.
* Used `cloud_free_trial_days` to get free trial days secret.
2023-12-10 15:18:01 -08:00
Tim Abbott
22d59f1132 typing notifications: Increase various time constants.
See https://chat.zulip.org/#narrow/stream/6-frontend/topic/typing.20notifications.20efficiency/near/1664991.

As detailed in that discussion,
`TYPING_STARTED_EXPIRY_PERIOD_MILLISECONDS` and
`TYPING_STARTED_WAIT_PERIOD_MILLISECONDS` are coupled constants, and
the impact of them being large is mainly that if a user closes their
computer or loses network in the middle of typing something (not
exactly a common occasion), then the client will suggest they kept on
typing longer than they in fact did.

There's a substantial decrease in resources consumed by this feature
associated with raising `TYPING_STARTED_WAIT_PERIOD_MILLISECONDS`, so
that at least seems worth doing.

Meanwhile, because TYPING_STOPPED_WAIT_PERIOD_MILLISECONDS measures
how long we should wait before deciding to stop suggesting a user is
still typing if they were previously typing a message but paused doing
so without closing the compose box (example causes being stepping away
from the computer, tabbing to go look something up, or just thinking
for a bit).

On the one hand, even the original 5 seconds is a fairly long time to
pause to think without touching the keyboard; on the other hand,
sitting with text you've written in the compose box is likely still a
quite high intent-to-send-soon state. Increasing this to 12 seconds
seems like a reasonable balance between being too trigger-happy here
here and avoiding someone who left their computer appearing like they
are still typing for a long time afterwards.
2023-12-09 17:05:59 -08:00
Prakhar Pratyush
d8cf12eaaa send_email: Improve configurability for outgoing email sender name.
Currently, the sender names for outgoing emails sent by Zulip
are hardcoded. It should be configurable for self-hosted systems.

This commit makes the 'Zulip' part a variable in the following
email sender names: 'Zulip Account Security', 'Zulip Digest',
and 'Zulip Notifications' by introducing a settings variable
'SERVICE_NAME' with the default value as f"{EXTERNAL_HOST} Zulip".

Fixes: #23857
2023-11-29 14:20:01 -08:00