Commit Graph

739 Commits

Author SHA1 Message Date
Alex Vandiver
544e8c569e install: Switch default to PostgreSQL 14. 2021-11-08 18:21:46 -08:00
Anders Kaseorg
f2a443a736 install-node: Upgrade Node.js from 14.18.1 to 16.13.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-11-05 17:34:13 -07:00
Anders Kaseorg
458844a2f5 install-yarn: Verify that the install location is /srv/zulip-yarn.
scripts.lib.node_cache expects Yarn to be in /srv/zulip-yarn, so if
it’s installed somewhere else, even if it’s the right version, we need
to reinstall it.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-11-03 16:49:58 -07:00
rht
bb8504d925 lint: Fix typos found by codespell. 2021-10-19 16:51:13 -07:00
Anders Kaseorg
291087d70c install-yarn: Upgrade Yarn from 1.22.11 to 1.22.17.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-10-17 07:15:09 -07:00
Anders Kaseorg
7df96b78c6 install-node: Upgrade Node.js from 14.17.6 to 14.18.1.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-10-17 07:15:09 -07:00
Anders Kaseorg
2f993f1a79 install-node: Stop using NVM.
NVM doesn’t check hashes or signatures and really just adds
complexity we don’t need.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-24 06:58:32 -07:00
Anders Kaseorg
902883d818 setup_venv: Skip virtualenv’s automatic download of setuptools.
It recently started failing on Debian 10 (buster).  We immediately
follow this by replacing these packages with our own versions from
pip.txt, anyway.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-23 14:29:04 -07:00
Anders Kaseorg
08e459b393 zulip_tools: Convert "".format to Python 3.6 f-strings.
Generated automatically by pyupgrade.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-22 13:58:46 -07:00
Anders Kaseorg
9bed17e0ab install-node: Upgrade Node.js from 14.17.5 to 14.17.6.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-13 10:12:43 -07:00
Gaurav Pandey
502697d239 docs: Add documentation for bullseye support.
The support for bullseye was added in #17951
but it was not documented as bullseye was
frozen and did not have proper configuration
files, hence wasn't documented.

Since now bullseye is released as a stable
version, it's support can be documented.
2021-09-09 11:05:16 -07:00
Anders Kaseorg
02582c6956 upgrade-zulip-from-git: Run git fetch with --prune.
This prevents upgrading to an obsolete version of a branch that has
been deleted or renamed.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-01 05:34:57 -07:00
Anders Kaseorg
3cb66d59ac install: Remove /dev/null redirect for zulip-puppet-apply.
The usual output from this command looks like

Notice: Compiled catalog for localhost in environment production in 2.33 seconds
Notice: /Stage[main]/Zulip::Apt_repository/Exec[setup_apt_repo]/returns: current_value 'notrun', should be ['0'] (noop)
Notice: Class[Zulip::Apt_repository]: Would have triggered 'refresh' from 1 event
Notice: Stage[main]: Would have triggered 'refresh' from 1 event
Notice: Applied catalog in 1.20 seconds

which doesn’t seem abnormally alarming, and hiding it makes failures
harder to diagnose.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-31 16:30:53 -07:00
Anders Kaseorg
7b2e585213 install-yarn: Upgrade Yarn from 1.22.10 to 1.22.11.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-23 12:33:27 -07:00
Anders Kaseorg
ebb8e9109c install-node: Upgrade Node.js from 14.17.3 to 14.17.5.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-23 12:29:04 -07:00
Anders Kaseorg
4206e5f00b python: Remove locally dead code.
These changes are all independent of each other; I just didn’t feel
like making dozens of commits for them.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg
5483ebae37 python: Convert "".format to Python 3.6 f-strings.
Generated automatically by pyupgrade.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-02 15:53:52 -07:00
Anders Kaseorg
ad5f0c05b5 python: Remove default "utf8" argument for encode(), decode().
Partially generated by pyupgrade.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-02 15:53:52 -07:00
Anders Kaseorg
1760897a8c python: Remove default "r" mode for open().
Generated automatically by pyupgrade.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-02 15:53:52 -07:00
Anders Kaseorg
3665deb93a python: Remove unnecessary intermediate lists.
Generated automatically by pyupgrade.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-02 15:53:52 -07:00
manavdesai27
572cef9a0f provision: Add support for Fedora 34. 2021-07-20 12:10:41 -07:00
Anders Kaseorg
47897c76a2 scripts: Use curl -f (--fail).
This makes curl exit with nonzero status on HTTP 4xx/5xx errors.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-13 16:47:49 -07:00
Alex Vandiver
c94bdd8534 zulip_tools: Find missing processes/groups in list_supervisor_processes.
Nonexistent processes and groups passed to `supervisortctl status` are
printed to STDOUT as follows:

```
$ supervisorctl status zulip-django nonexistent-process nonexistent-group:*
nonexistent-process: ERROR (no such process)
nonexistent-group: ERROR (no such group)
zulip-django                     RUNNING   pid 16043, uptime 17:31:31
```

On supervisor 4 and above, this exits with an exit code of 4;
previously, it returned exit code 0.  Ubuntu 18.04 has version 3.3.1,
and Ubuntu 20.04 has version 4.1.0.

Skip any lines with `ERROR (no such ...)`, and accept exit code 4 from
`supervisorctl status`.
2021-07-09 10:04:53 -07:00
Alex Vandiver
85a9c0982a zulip_tools: Extract out list_supervisor_processes. 2021-07-09 10:04:53 -07:00
Anders Kaseorg
d83c91526b install-node: Upgrade Node.js from 14.17.0 to 14.17.3.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-05 14:51:24 -07:00
Anders Kaseorg
0ba9114c22 install-yarn: Rewrite Yarn installer.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-06-27 16:58:17 -07:00
Anders Kaseorg
91bfebca7d install: Replace wget with curl.
curl uses Happy Eyeballs to avoid long timeouts on systems with broken
IPv6.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-06-25 09:05:07 -07:00
Anders Kaseorg
3b60b25446 ci: Remove bullseye hack.
base-files 11.1 marked bullseye as Debian 11 in /etc/os-release.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-06-24 14:35:51 -07:00
Anders Kaseorg
bf361e9951 ci: Remove uses of VERSION_CODENAME.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-06-24 14:35:51 -07:00
Tim Abbott
28d49edee3 script: Add --no-headings option to purge-old-deployments.
This parameter is somewhat useful, and adding this also fixes a
regression where purge-old-deployments would crash since the changes
around c5580607a7 because of
inconsistent supported args lists.
2021-06-17 15:49:23 -07:00
Mateusz Mandera
06c0a29e47 email-mirror-postfix: Choose scheme based on http_only config.
Fixes #16659.
If the server is behind a reverse proxy with http_only=True, the
requests made by email-mirror-postfix need to use http, as https
doesn't work.
2021-06-17 09:06:09 -07:00
Alex Vandiver
d51272cc3d puppet: Remove zulip_deliver_scheduled_* from zulip-workers:*.
Staging and other hosts that are `zulip::app_frontend_base` but not
`zulip::app_frontend_once` do not have a
/etc/supervisor/conf.d/zulip/zulip-once.conf and as such do not have
`zulip_deliver_scheduled_emails` or `zulip_deliver_scheduled_messages`
and thus supervisor will fail to reload.

Making the contents of `zulip-workers` contingent on if the server is
_also_ a `-once` server is complicated, and would involve using Concat
fragments, which severely limit readability.

Instead, expel those two from `zulip-workers`; this is somewhat
reasonable, since they are use an entirely different codepath from
zulip_events_*, using the database rather than RabbitMQ for their
queuing.
2021-06-14 17:12:59 -07:00
Riken Shah
45af71e33b clean_unused_caches: Allow the main function to accept Namespace args.
This commit will allow us to pass the arguments in the
'clean...' functions when calling the `main` function (in
`provision`). It also changes args parsing
function location to `if __name__ == "__main__"` block as
we wouldn't need it to parse args when we call the
function.
2021-06-12 07:28:16 -07:00
Riken Shah
4f54e15993 refactor: Convert clean-unused-caches toclean_unused_caches.py.
We convert the `clean-unused-caches` script to a
python file so we can run it in provision by importing it
instead of running the script, hence saving some time.
2021-06-12 07:28:16 -07:00
Anders Kaseorg
d8cb418586 zulip_tools: Flush ‘set -x’-style messages in run.
Otherwise they often get buffered until after the command actually
runs.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-06-09 14:05:31 -07:00
Anders Kaseorg
342834ee9c python: Simplify stdio flushing using print(…, flush=True).
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-06-09 14:05:31 -07:00
Anders Kaseorg
bc169d63a7 install-node: Upgrade Node.js from 14.16.1 to 14.17.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-06-08 16:02:12 -07:00
Alex Vandiver
e080a05b05 node_cache: Serialize to structured data before hashing.
Appending data back-to-back without serializing it loses the
information about where the breaks between them lie, which can lead to
different inputs having the same hash.
2021-05-27 22:47:56 -07:00
Alex Vandiver
87a109e3e0 puppet: Pull in pinned puppet modules.
Using puppet modules from the puppet forge judiciously will allow us
to simplify the configuration somewhat; this specifically pulls in the
stdlib module, which we were already using parts of.
2021-05-27 21:14:48 -07:00
Alex Vandiver
f3eea72c2a setup: Merge multiple setup-apt-repo scripts into one.
This moves the `.asc` files into subdirectories, and writes out the
according `.list` files into them.  It moves from templates to
written-out `.list` files for clarity and ease of
implementation (Debian and Ubuntu need different templates for
`zulip`), and as a way of making explicit which releases are supported
for each list.  For the special-case of the PGroonga signing key, we
source an additional file within the directory.

This simplifies the process for adding another class of `.list` file.
2021-05-26 14:42:29 -07:00
Adam Birds
4539899cae installer: Add support for custom database user and dbname.
Add support for custom database names and database users, which can be
set with the `--postgresql-database-name` and
`--postgresql-database-user` install script options. If these
parameters aren't provided, then the defaults remain "zulip".

Fixes #17662.

Co-authored-by: Alex Vandiver <alexmv@zulip.com>
2021-05-25 13:56:05 -07:00
Alex Vandiver
7ff3c9f966 upgrade-zulip: Support arbitrary database user and dbname.
Co-authored-by: Adam Birds <adam.birds@adbwebdesigns.co.uk>
2021-05-25 13:56:05 -07:00
Adam Birds
21cc186105 installer: Add run_psql_as_postgres function zulip_tools.py.
Add a helper `run_psql_as_postgres` function in
`scripts/lib/zulip_tools.py`.  This is preparatory refactoring for the
work to add custom database and user names.
2021-05-24 16:58:11 -07:00
Alex Vandiver
81644f110e install: $ZULIP_ADMINISTRATOR may be unset for non-frontend hosts. 2021-05-23 13:29:23 -07:00
Anders Kaseorg
09f6ba1971 install: Run git config commands from a known readable cwd.
Fixes this error when running the installer from a directory that
isn’t world-readable:

+ su zulip -c 'git config --global user.email anders@zulip.com'
fatal: cannot come back to cwd: Permission denied

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-13 22:00:29 -07:00
Tim Abbott
de47feab43 scripts: Fix check for services running when upgrading.
When upgrading from a pre-4.0 release, scripts/stop-server logic would
check whether supervisord configuration files were present to
determine what it needed to restart, but only considered paths to
those files that are introduced in Zulip 4.0.

Fixed #18493.
2021-05-13 18:57:19 -07:00
Anders Kaseorg
3f83b843c2 upgrade-zulip-from-git: Create deployment directories with git worktree.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-13 13:42:05 -07:00
Tim Abbott
03420831b0 upgrade-zulip-from-git: Fetch tags from upstream repository.
This ensures that the `git describe` queries that we run for caching
Zulip's Git version are guaranteed to include recent releases.

This change ensures that we have accurate output even if we're pointed
at a fork of Zulip that never updates its tags.

Additionally, it will make it possible to record the
`git merge-base upstream/master` in future commits.

Note that because we run this code before unpacking the new version,
the pre-upgrade version of this code runs.

As a result, we cannot assume that the upstream repository exists.
2021-05-13 11:17:25 -07:00
Alex Vandiver
3ccb77da74 install: Tell NVM to not change $PATH earlier.
This removes a possible window where an installer error could leave
`nvm` in a state where it had prepended the full path to the
newly-installed `npm` to `$PATH`; we would like to avoid `nvm`
fiddling with path whenever possible (ref ebe930ab2c).
2021-05-11 11:25:34 -10:00
Anders Kaseorg
9ba48c4ed3 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-07 22:42:39 -07:00