requirements: Migrate to uv.

https://docs.astral.sh/uv/

Signed-off-by: Anders Kaseorg <anders@zulip.com>

docs/development/request-remote.md

@@ -60,7 +60,7 @@ Once your remote dev instance is ready:
   `ssh zulipdev@<username>.zulipdev.org` on the command line
   (Terminal for macOS and Linux, Bash for Git on Windows).
 - There is no password; your account is configured to use your SSH keys.
-- Once you log in, you should see `(zulip-py3-venv) ~$`.
+- Once you log in, you should see `(zulip-server) ~$`.
 - To start the dev server, `cd zulip` and then run `./tools/run-dev`.
 - While the dev server is running, you can see the Zulip server in your browser
   at http://zulip.username.zulipdev.org:9991.

docs/development/setup-advanced.md

@@ -54,7 +54,7 @@ Zulip and run the following commands:
 ```bash
 # From inside a clone of zulip.git:
 ./tools/provision
-source /srv/zulip-py3-venv/bin/activate
+source .venv/bin/activate
 ./tools/run-dev  # starts the development server
 ```
 
@@ -212,8 +212,8 @@ expected.
 1. Set the `EXTERNAL_HOST` environment variable.
 
    ```console
-   (zulip-py3-venv) vagrant@ubuntu-18:/srv/zulip$ export EXTERNAL_HOST="$(hostname -I | xargs):9991"
-   (zulip-py3-venv) vagrant@ubuntu-18:/srv/zulip$ echo $EXTERNAL_HOST
+   (zulip-server) vagrant@ubuntu-18:/srv/zulip$ export EXTERNAL_HOST="$(hostname -I | xargs):9991"
+   (zulip-server) vagrant@ubuntu-18:/srv/zulip$ echo $EXTERNAL_HOST
    ```
 
    The output will be like:
@@ -234,7 +234,7 @@ expected.
 1. You should now be able to start the Zulip development server.
 
    ```console
-   (zulip-py3-venv) vagrant@ubuntu-18:/srv/zulip$ ./tools/run-dev
+   (zulip-server) vagrant@ubuntu-18:/srv/zulip$ ./tools/run-dev
    ```
 
    The output will look like:

docs/development/setup-recommended.md

@@ -293,7 +293,7 @@ simply click **Allow access**.)
 $ # Install/update the Zulip development environment
 $ ./tools/provision
 $ # Enter the Zulip Python environment
-$ source /srv/zulip-py3-venv/bin/activate
+$ source .venv/bin/activate
 $ # Start the development server
 $ ./tools/run-dev
 ```
@@ -681,11 +681,11 @@ Alternatively, you can use a command to terminate/shutdown your WSL2 environment
 
 On Windows with WSL 2, to resume developing you just need to open a new Git
 BASH window. Then change into your `zulip` folder and verify the Python
-environment was properly activated (you should see `(zulip-py3-venv)`). If the
-`(zulip-py3-venv)` part is missing, run:
+environment was properly activated (you should see `(zulip-server)`). If the
+`(zulip-server)` part is missing, run:
 
 ```console
-$ source /srv/zulip-py3-venv/bin/activate
+$ source .venv/bin/activate
 ```
 
 :::
@@ -765,7 +765,7 @@ When reporting your issue, please include the following information:
 The output of `tools/diagnose` run inside the Vagrant guest is also
 usually helpful.
 
-#### Vagrant guest doesn't show (zulip-py3-venv) at start of prompt
+#### Vagrant guest doesn't show (zulip-server) at start of prompt
 
 This is caused by provisioning failing to complete successfully. You
 can see the errors in `var/log/provision.log`; it should end with
@@ -1009,13 +1009,13 @@ Once you've provisioned successfully, you'll get output like this:
 
 ```console
 Zulip development environment setup succeeded!
-(zulip-py3-venv) vagrant@vagrant:/srv/zulip$
+(zulip-server) vagrant@vagrant:/srv/zulip$
 ```
 
-If the `(zulip-py3-venv)` part is missing, this is because your
+If the `(zulip-server)` part is missing, this is because your
 installation failed the first time before the Zulip virtualenv was
 created. You can fix this by just closing the shell and running
-`vagrant ssh` again, or using `source /srv/zulip-py3-venv/bin/activate`.
+`vagrant ssh` again, or using `source .venv/bin/activate`.
 
 Finally, if you encounter any issues that weren't caused by your
 Internet connection, please report them! We try hard to keep Zulip

docs/development/setup/vagrant-halt.md

@@ -11,7 +11,7 @@ From the window where run-dev is running:
 2016-05-04 18:33:13,330 INFO     127.0.0.1       GET     200  92ms /register/ (unauth@zulip via ?)
 ^C
 KeyboardInterrupt
-(zulip-py3-venv) vagrant@vagrant:/srv/zulip$ exit
+(zulip-server) vagrant@vagrant:/srv/zulip$ exit
 logout
 Connection to 127.0.0.1 closed.
 $

docs/development/setup/vagrant-resume.md

@@ -7,5 +7,5 @@ Zulip server:
 $ vagrant up
 $ vagrant ssh
 
-(zulip-py3-venv) vagrant@vagrant:/srv/zulip$ ./tools/run-dev
+(zulip-server) vagrant@vagrant:/srv/zulip$ ./tools/run-dev
 ```

docs/development/setup/vagrant-ssh.md

@@ -14,14 +14,14 @@ Welcome to Ubuntu 22.04.3 LTS (GNU/Linux 5.15.0-92-generic x86_64)
 Congrats, you're now inside the Zulip development environment!
 
 You can confirm this by looking at the command prompt, which starts
-with `(zulip-py3-venv)vagrant@`. If it just starts with `vagrant@`, your
+with `(zulip-server) vagrant@`. If it just starts with `vagrant@`, your
 provisioning failed and you should look at the
 [troubleshooting section](/development/setup-recommended.md#troubleshooting-and-common-errors).
 
 Next, start the Zulip server:
 
 ```console
-(zulip-py3-venv) vagrant@vagrant:/srv/zulip$ ./tools/run-dev
+(zulip-server) vagrant@vagrant:/srv/zulip$ ./tools/run-dev
 ```
 
 You will see something like:

docs/documentation/overview.md

@@ -71,11 +71,10 @@ and looking at the content on the GitHub web UI, since GitHub renders
 Markdown, though that won't be as faithful as the `make html`
 approach or the preview build.
 
-When editing dependencies for the Zulip documentation, you should edit
-`requirements/docs.in` and then run `tools/update-locked-requirements`
-which updates docs.txt file (which is used by ReadTheDocs to build the
-Zulip developer documentation, without installing all of Zulip's
-dependencies).
+We manage Python requirements for the documentation build in the `docs` uv
+[group](https://docs.astral.sh/uv/concepts/projects/dependencies/#dependency-groups),
+which is used by our ReadTheDocs build configuration in
+[`.readthedocs.yaml`](https://docs.readthedocs.com/platform/stable/config-file/v2.html).
 
 ## Core website documentation
 

docs/subsystems/api-release-checklist.md

@@ -24,7 +24,7 @@ checklist of things one must do before making a PyPI release:
    [Zulip server repo][zulip-repo] to render the interactive bots on
    [Zulip's integrations page](https://zulip.com/integrations/). The server
    repo installs the `zulip_bots` package
-   [directly from the GitHub repository][requirements-link] so that this extra
+   directly from the GitHub repository so that this extra
    content is included in its installation of the package.
 
 3. Follow PyPI's instructions in
@@ -57,9 +57,9 @@ Now it is time to [update the dependencies](dependencies) in the
    most cases.
 
 2. Update the release tags in the Git URLs for `zulip` and `zulip_bots` in
-   `requirements/common.in`.
+   `pyproject.toml`.
 
-3. Run `tools/update-locked-requirements` to update the rest of the requirements files.
+3. Run `uv lock` to update the Python lock file.
 
 4. Commit your changes and submit a PR! **Note**: See
    [this example commit][example-zulip-commit] to get an idea of what the final change
@@ -83,7 +83,6 @@ update to dependencies required in the [Zulip server repo][zulip-repo].
 [zulip-package]: https://github.com/zulip/python-zulip-api/tree/main/zulip
 [zulip-bots-package]: https://github.com/zulip/python-zulip-api/tree/main/zulip_bots
 [zulip-botserver-package]: https://github.com/zulip/python-zulip-api/tree/main/zulip_botserver
-[requirements-link]: https://github.com/zulip/zulip/blob/main/requirements/common.in#L116
 [generating-dist-archives]: https://packaging.python.org/en/latest/tutorials/packaging-projects/#generating-distribution-archives
 [upload-dist-archives]: https://packaging.python.org/en/latest/tutorials/packaging-projects/#uploading-the-distribution-archives
 [install-pkg]: https://packaging.python.org/en/latest/tutorials/packaging-projects/#installing-your-newly-uploaded-package

docs/subsystems/dependencies.md

@@ -125,70 +125,12 @@ Ubuntu 22.04 being the platform requiring 3.10 support. The comments
 in `.github/workflows/zulip-ci.yml` document the Python versions used
 by each supported platform.
 
-We manage Python packages via the Python-standard `requirements.txt`
-system and virtualenvs, but there’s a number of interesting details
-about how Zulip makes this system work well for us that are worth
-highlighting. The system is largely managed by the code in
-`scripts/lib/setup_venv.py`
+We manage third-party Python packages using [uv](https://docs.astral.sh/uv/),
+with our requirements listed in
+[pyproject.toml](https://docs.astral.sh/uv/concepts/projects/layout/#the-pyprojecttoml),
+and locked versions stored in
+[`uv.lock`](https://docs.astral.sh/uv/concepts/projects/layout/#the-lockfile).
 
-- **Using `pip` to manage dependencies**. This is standard in the
-  Python ecosystem, and means we only need to record a list of
-  versions in a `requirements.txt` file to declare what we're using.
-  Since we have a few different installation targets, we maintain
-  several `requirements.txt` format files in the `requirements/`
-  directory (e.g., `dev.in` for development, `prod.in` for
-  production, `docs.in` for ReadTheDocs, `common.in` for the vast
-  majority of packages common to prod and development, etc.). We use
-  `pip install --no-deps` to ensure we only install the packages we
-  explicitly declare as dependencies.
-- **virtualenv with pinned versions**. For a large application like
-  Zulip, it is important to ensure that we're always using consistent,
-  predictable versions of all of our Python dependencies. To ensure
-  this, we install our dependencies in a [virtualenv][] that contains
-  only the packages and versions that Zulip needs, and we always pin
-  exact versions of our dependencies in our `requirements.txt` files.
-  We pin exact versions, not minimum versions, so that installing
-  Zulip won't break if a dependency makes a buggy release. A side
-  effect is that it's easy to debug problems caused by dependency
-  upgrades, since we're always doing those upgrades with an explicit
-  commit updating the `requirements/` directory.
-- **Pinning versions of indirect dependencies**. We "pin" or "lock"
-  the versions of our indirect dependencies files with
-  `tools/update-locked-requirements` (powered by `pip-compile`). What
-  this means is that we have some "source" requirements files, like
-  `requirements/common.in`, that declare the packages that Zulip
-  depends on directly. Those packages have their own recursive
-  dependencies. When adding or removing a dependency from Zulip, one
-  simply edits the appropriate "source" requirements files, and then
-  runs `tools/update-locked-requirements`. That tool will use
-  `pip-compile` to generate the locked requirements files like
-  `prod.txt`, `dev.txt` etc files that explicitly declare versions of
-  all of Zulip's recursive dependencies. For indirect dependencies
-  (i.e. dependencies not explicitly declared in the source
-  requirements files), it provides helpful comments explaining which
-  direct dependency (or dependencies) needed that indirect dependency.
-  The process for using this system is documented in more detail in
-  `requirements/README.md`.
-- **Caching of virtualenvs and packages**. To make updating the
-  dependencies of a Zulip installation efficient, we maintain a cache
-  of virtualenvs named by the hash of the relevant `requirements.txt`
-  file (`scripts/lib/hash_reqs.py`). These caches live under
-  `/srv/zulip-venv-cache/<hash>`. That way, when re-provisioning a
-  development environment or deploying a new production version with
-  the same Python dependencies, no downloading or installation is
-  required: we just use the same virtualenv. When the only changes
-  are upgraded versions, we'll use [virtualenv-clone][] to clone the
-  most similar existing virtualenv and then just upgrade the packages
-  needed, making small version upgrades extremely efficient. And
-  finally, we use `pip`'s built-in caching to ensure that a specific
-  version of a specific package is only downloaded once.
-- **Garbage-collecting caches**. We have a tool,
-  `scripts/lib/clean_venv_cache.py`, which will clean old cached
-  virtualenvs that are no longer in use. In production, the algorithm
-  preserves recent virtualenvs as well as those in use by any current
-  production deployment directory under `/home/zulip/deployments/`.
-  This helps ensure that a Zulip installation doesn't leak large
-  amounts of disk over time.
 - **Scripts**. Often, we want a script running in production to use
   the Zulip virtualenv. To make that work without a lot of duplicated
   code, we have a helpful function,
@@ -203,15 +145,7 @@ highlighting. The system is largely managed by the code in
   `ignore_missing_imports` for the new library. See
   [our mypy docs][mypy-docs] for more details.
 
-### Upgrading packages
-
-See the [README][requirements-readme] file in `requirements/` directory
-to learn how to upgrade a single Python package.
-
 [mypy-docs]: ../testing/mypy.md
-[requirements-readme]: https://github.com/zulip/zulip/blob/main/requirements/README.md#requirements
-[stack-overflow]: https://askubuntu.com/questions/8653/how-to-keep-processes-running-after-ending-ssh-session
-[caching]: https://help.github.com/en/articles/caching-your-github-password-in-git
 
 ## JavaScript and other frontend packages
 

docs/subsystems/django-upgrades.md

@@ -15,7 +15,7 @@ new major versions of Django. Here are the steps:
   much of the changes for the migration as we can independently from
   the big cutover.
 - Check the version support of the third-party Django packages we use
-  (`git grep django requirements/` to see a list), upgrade any as
+  (`git grep django pyproject.toml` to see a list), upgrade any as
   needed and file bugs upstream for any that lack support. Look into
   fixing said bugs.
 - Look at the pieces of Django code that we've copied and then

docs/testing/continuous-integration.md

@@ -124,6 +124,6 @@ would have used had the cache not existed. In practice, bugs are
 always possible, so be mindful of this possibility.
 
 A consequence of this caching is that test jobs for branches which
-modify `package.json`, `requirements/`, and other key dependencies
+modify `package.json`, `pyproject.toml`, and other key dependencies
 will be significantly slower than normal, because they won't get to
 benefit from the cache.

docs/testing/testing.md

@@ -70,11 +70,6 @@ Additionally, Zulip also has about a dozen smaller tests suites:
 - `tools/test-api`: Tests that the API documentation at `/api`
   actually works; the actual code for this is defined in
   `zerver/openapi/python_examples.py`.
-- `test-locked-requirements`: Verifies that developers didn't forget
-  to run `tools/update-locked-requirements` after modifying
-  `requirements/*.in`. See
-  [our dependency documentation](../subsystems/dependencies.md) for
-  details on the system this is verifying.
 - `tools/check-capitalization`: Checks whether translated strings (aka
   user-facing strings) correctly follow Zulip's capitalization
   conventions. This requires some maintenance of an exclude list