paulmataruso/zulip
1
0
Fork 0
mirror of https://github.com/zulip/zulip.git synced 2025-11-01 20:44:04 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: Format Markdown with Prettier.

Browse Source 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit a4dbc1edd4)
This commit is contained in:
Anders Kaseorg
2021-08-20 13:54:08 -07:00
committed by Tim Abbott
parent aa6e70382d
commit d81ce3ba76
106 changed files with 1912 additions and 1742 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
docs/contributing/accessibility.md
View File

@@ -79,7 +79,6 @@ possible, the following resources may be useful.
- The [MDN page on accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility)
- The [Open edX Accessibility Guidelines][openedx-guidelines] for developers
[chrome-webstore]: https://chrome.google.com/webstore/detail/accessibility-developer-t/fpkknkljclfencbdbgkenhalefipecmb
[openedx-guidelines]: https://edx.readthedocs.io/projects/edx-developer-guide/en/latest/conventions/accessibility.html
[accessibility-issues]: https://github.com/zulip/zulip/issues?q=is%3Aissue+is%3Aopen+label%3A%22area%3A%20accessibility%22

7
docs/contributing/bug-reports.md
View File

@@ -9,14 +9,15 @@ Please include these elements in your bug report to make it easier for us to hel
- Steps to take in order to reproduce the buggy behavior
- Whether you are using Zulip in production or in the development
environment, and whether these are old versions
environment, and whether these are old versions
- Whether you are using the web app, a desktop app or a mobile device
to access Zulip
to access Zulip
- Any additional information that would help: screenshots, GIFs, a
pastebin of the error log
pastebin of the error log
Further reading:
- [How to write a bug report that will make your engineers love you](https://testlio.com/blog/the-ideal-bug-report/)
- [How to Report Bugs Effectively](https://www.chiark.greenend.org.uk/~sgtatham/bugs.html)

35
docs/contributing/code-reviewing.md
View File

@@ -10,6 +10,7 @@ When you send a PR, try to think of a good person to review it --
outside of the handful of people who do a ton of reviews -- and
`@`-mention them with something like "`@person`, would you review
this?". Good choices include
- someone based in your timezone or a nearby timezone
- people working on similar things, or in a loosely related area
@@ -106,7 +107,7 @@ sooner is better.
## Things to look for
- *The CI build.* The tests need to pass. One can investigate
- _The CI build._ The tests need to pass. One can investigate
any failures and figure out what to fix by clicking on a red X next
to the commit hash or the Detail links on a pull request. (Example:
in [#17584](https://github.com/zulip/zulip/pull/17584),
@@ -120,25 +121,25 @@ sooner is better.
See our docs on [continuous integration](../testing/continuous-integration.md)
to learn more.
- *Technical design.* There are a lot of considerations here:
- _Technical design._ There are a lot of considerations here:
security, migration paths/backwards compatibility, cost of new
dependencies, interactions with features, speed of performance, API
changes. Security is especially important and worth thinking about
carefully with any changes to security-sensitive code like views.
- *User interface and visual design.* If frontend changes are
- _User interface and visual design._ If frontend changes are
involved, the reviewer will check out the code, play with the new
UI, and verify it for both quality and consistency with the rest of
the Zulip UI. We highly encourage posting screenshots to save
reviewers time in getting a feel for what the feature looks like --
you'll get a quicker response that way.
- *Error handling.* The code should always check for invalid user
- _Error handling._ The code should always check for invalid user
input. User-facing error messages should be clear and when possible
be actionable (it should be obvious to the user what they need to do
in order to correct the problem).
- *Testing.* The tests should validate that the feature works
- _Testing._ The tests should validate that the feature works
correctly, and specifically test for common error conditions, bad
user input, and potential bugs that are likely for the type of
change being made. Tests that exclude whole classes of potential
@@ -147,16 +148,16 @@ sooner is better.
Markdown processors](../subsystems/markdown.md), or the `GetEventsTest` test for
buggy race condition handling).
- *Translation.* Make sure that the strings are marked for
- _Translation._ Make sure that the strings are marked for
[translation].
- *Clear function, argument, variable, and test names.* Every new
- _Clear function, argument, variable, and test names._ Every new
piece of Zulip code will be read many times by other developers, and
future developers will grep for relevant terms when researching a
problem, so it's important that variable names communicate clearly
the purpose of each piece of the codebase.
- *Duplicated code.* Code duplication is a huge source of bugs in
- _Duplicated code._ Code duplication is a huge source of bugs in
large projects and makes the codebase difficult to understand, so we
avoid significant code duplication wherever possible. Sometimes
avoiding code duplication involves some refactoring of existing
@@ -164,32 +165,32 @@ sooner is better.
commits (not squashed into other changes or left as a thing to do
later). That series of commits can be in the same pull request as
the feature that they support, and we recommend ordering the history
of commits so that the refactoring comes *before* the feature. That
of commits so that the refactoring comes _before_ the feature. That
way, it's easy to merge the refactoring (and minimize risk of merge
conflicts) if there are still user experience issues under
discussion for the feature itself.
- *Completeness.* For refactorings, verify that the changes are
- _Completeness._ For refactorings, verify that the changes are
complete. Usually one can check that efficiently using `git grep`,
and it's worth it, as we very frequently find issues by doing so.
- *Documentation updates.* If this changes how something works, does it
- _Documentation updates._ If this changes how something works, does it
update the documentation in a corresponding way? If it's a new
feature, is it documented, and documented in the right place?
- *Good comments.* It's often worth thinking about whether explanation
- _Good comments._ It's often worth thinking about whether explanation
in a commit message or pull request discussion should be included in
a comment, `/docs`, or other documentation. But it's better yet if
verbose explanation isn't needed. We prefer writing code that is
readable without explanation over a heavily commented codebase using
lots of clever tricks.
- *Coding style.* See the Zulip [code-style] documentation for
- _Coding style._ See the Zulip [code-style] documentation for
details. Our goal is to have as much of this as possible verified
via the linters and tests, but there's always going to be unusual
forms of Python/JavaScript style that our tools don't check for.
- *Clear commit messages.* See the [Zulip version
- _Clear commit messages._ See the [Zulip version
control][commit-messages] documentation for details on what we look
for.
@@ -197,15 +198,15 @@ sooner is better.
Some points specific to the Zulip server codebase:
- *Testing -- Backend.* We are trying to maintain ~100% test coverage
- _Testing -- Backend._ We are trying to maintain ~100% test coverage
on the backend, so backend changes should have negative tests for
the various error conditions.
- *Testing -- Frontend.* If the feature involves frontend changes,
- _Testing -- Frontend._ If the feature involves frontend changes,
there should be frontend tests. See the [test
writing][test-writing] documentation for more details.
- *mypy annotations.* New functions should be annotated using [mypy]
- _mypy annotations._ New functions should be annotated using [mypy]
and existing annotations should be updated. Use of `Any`, `ignore`,
and unparameterized containers should be limited to cases where a
more precise type cannot be specified.

88
docs/contributing/code-style.md
View File

@@ -48,15 +48,15 @@ The Vagrant setup process runs this for you.
`lint` runs many lint checks in parallel, including
- JavaScript ([ESLint](https://eslint.org/),
[Prettier](https://prettier.io/))
- Python ([mypy](http://mypy-lang.org/),
[Pyflakes](https://pypi.python.org/pypi/pyflakes),
[Black](https://github.com/psf/black),
[isort](https://pycqa.github.io/isort/))
- templates
- Puppet configuration
- custom checks (e.g. trailing whitespace and spaces-not-tabs)
- JavaScript ([ESLint](https://eslint.org/),
[Prettier](https://prettier.io/))
- Python ([mypy](http://mypy-lang.org/),
[Pyflakes](https://pypi.python.org/pypi/pyflakes),
[Black](https://github.com/psf/black),
[isort](https://pycqa.github.io/isort/))
- templates
- Puppet configuration
- custom checks (e.g. trailing whitespace and spaces-not-tabs)
## Secrets
@@ -143,9 +143,9 @@ some_objs = UserProfile.objects.get(id=17)
assert obj.id in set([o.id for i in some_objs])
```
### user\_profile.save()
### user_profile.save()
You should always pass the update\_fields keyword argument to .save()
You should always pass the update_fields keyword argument to .save()
when modifying an existing Django model object. By default, .save() will
overwrite every value in the column, which results in lots of race
conditions where unrelated changes made by one thread can be
@@ -155,7 +155,7 @@ object before the first thread wrote out its change.
### Using raw saves to update important model objects
In most cases, we already have a function in zerver/lib/actions.py with
a name like do\_activate\_user that will correctly handle lookups,
a name like do_activate_user that will correctly handle lookups,
caching, and notifying running browsers via the event system about your
change. So please check whether such a function exists before writing
new code to modify a model object, since your new code has a good chance
@@ -172,6 +172,7 @@ libraries as `from datetime import datetime, timezone` and
`from django.utils.timezone import now as timezone_now`.
Use:
- `timezone_now()` to get a datetime when Django is available, such as
in `zerver/`.
- `datetime.now(tz=timezone.utc)` when Django is not available, such as
@@ -189,6 +190,7 @@ parameter, `datetime.utcnow()` and `datetime.utcfromtimestamp()`, and
the end.
Additional notes:
- Especially in scripts and puppet configuration where Django is not
available, using `time.time()` to get timestamps can be cleaner than
dealing with datetimes.
@@ -295,11 +297,11 @@ We used to favor attaching behaviors in templates like so:
but there are some reasons to prefer attaching events using jQuery code:
- Potential huge performance gains by using delegated events where
possible
- When calling a function from an `onclick` attribute, `this` is not
bound to the element like you might think
- jQuery does event normalization
- Potential huge performance gains by using delegated events where
possible
- When calling a function from an `onclick` attribute, `this` is not
bound to the element like you might think
- jQuery does event normalization
Either way, avoid complicated JavaScript code inside HTML attributes;
call a helper function instead.
@@ -322,34 +324,34 @@ type changes in the future.
### Python
- Our Python code is formatted with
[Black](https://github.com/psf/black) and
[isort](https://pycqa.github.io/isort/). The [linter
tool](../testing/linters.md) enforces this by running Black and
isort in check mode, or in write mode with
`tools/lint --only=black,isort --fix`. You may find it helpful to
[integrate
Black](https://black.readthedocs.io/en/stable/editor_integration.html)
and
[isort](https://pycqa.github.io/isort/#installing-isorts-for-your-preferred-text-editor)
with your editor.
- Don't put a shebang line on a Python file unless it's meaningful to
run it as a script. (Some libraries can also be run as scripts, e.g.
to run a test suite.)
- Scripts should be executed directly (`./script.py`), so that the
interpreter is implicitly found from the shebang line, rather than
explicitly overridden (`python script.py`).
- Put all imports together at the top of the file, absent a compelling
reason to do otherwise.
- Unpacking sequences doesn't require list brackets:
- Our Python code is formatted with
[Black](https://github.com/psf/black) and
[isort](https://pycqa.github.io/isort/). The [linter
tool](../testing/linters.md) enforces this by running Black and
isort in check mode, or in write mode with
`tools/lint --only=black,isort --fix`. You may find it helpful to
[integrate
Black](https://black.readthedocs.io/en/stable/editor_integration.html)
and
[isort](https://pycqa.github.io/isort/#installing-isorts-for-your-preferred-text-editor)
with your editor.
- Don't put a shebang line on a Python file unless it's meaningful to
run it as a script. (Some libraries can also be run as scripts, e.g.
to run a test suite.)
- Scripts should be executed directly (`./script.py`), so that the
interpreter is implicitly found from the shebang line, rather than
explicitly overridden (`python script.py`).
- Put all imports together at the top of the file, absent a compelling
reason to do otherwise.
- Unpacking sequences doesn't require list brackets:
```python
[x, y] = xs # unnecessary
x, y = xs # better
```
```python
[x, y] = xs # unnecessary
x, y = xs # better
```
- For string formatting, use `x % (y,)` rather than `x % y`, to avoid
ambiguity if `y` happens to be a tuple.
- For string formatting, use `x % (y,)` rather than `x % y`, to avoid
ambiguity if `y` happens to be a tuple.
### Tests

46
docs/contributing/gsoc-ideas.md
View File

@@ -138,19 +138,19 @@ Also, you're going to find that people give you links to pages that
answer your questions. Here's how that often works:
1. you [try to solve your problem until you get stuck, including
looking through our code and our documentation, then start formulating
your request for
help](https://blogs.akamai.com/2013/10/you-must-try-and-then-you-must-ask.html)
looking through our code and our documentation, then start formulating
your request for
help](https://blogs.akamai.com/2013/10/you-must-try-and-then-you-must-ask.html)
1. you ask your question
1. someone directs you to a document
1. you go read that document, and try to use it to answer your question
1. you find you are confused about a new thing
1. you ask another question
1. now that you have demonstrated that you have the ability to read,
think, and learn new things, someone has a longer talk with you to
answer your new specific question
think, and learn new things, someone has a longer talk with you to
answer your new specific question
1. you and the other person collaborate to improve the document that you
read in step 3 :-)
read in step 3 :-)
This helps us make a balance between person-to-person discussion and
documentation that everyone can read, so we save time answering common
@@ -484,17 +484,17 @@ CSS](https://github.com/zulip/zulip/).
language and API design. Expert: Depends on language.
- Develop [**@zulipbot**](https://github.com/zulip/zulipbot), the GitHub
workflow bot for the Zulip organization and its repositories. By utilizing the
[GitHub API](https://developer.github.com/v3/),
[**@zulipbot**](https://github.com/zulipbot) improves the experience of Zulip
contributors by managing the issues and pull requests in the Zulip repositories,
such as assigning issues to contributors and appropriately labeling issues with
their current status to help contributors gain a better understanding of which
issues are being worked on. Since the project is in its early stages of
development, there are a variety of possible tasks that can be done, including
adding new features, writing unit tests and creating a testing framework, and
writing documentation. **Skills required**: Node.js, ECMAScript 6, and API
experience. Experts: Cynthia Lin, Joshua Pan.
workflow bot for the Zulip organization and its repositories. By utilizing the
[GitHub API](https://developer.github.com/v3/),
[**@zulipbot**](https://github.com/zulipbot) improves the experience of Zulip
contributors by managing the issues and pull requests in the Zulip repositories,
such as assigning issues to contributors and appropriately labeling issues with
their current status to help contributors gain a better understanding of which
issues are being worked on. Since the project is in its early stages of
development, there are a variety of possible tasks that can be done, including
adding new features, writing unit tests and creating a testing framework, and
writing documentation. **Skills required**: Node.js, ECMAScript 6, and API
experience. Experts: Cynthia Lin, Joshua Pan.
### React Native mobile app
@@ -522,11 +522,11 @@ summer. We'd love to have multiple students working on this area if
we have enough strong applicants.
**Skills required**: Strong programming experience, especially in
reading the documentation of unfamiliar projects and communicating
what you learned. JavaScript and React experience are great pluses,
as are iOS or Android development/design experience is useful as
well. You'll need to learn React Native as part of getting
involved. There's tons of good online tutorials, courses, etc.
reading the documentation of unfamiliar projects and communicating
what you learned. JavaScript and React experience are great pluses,
as are iOS or Android development/design experience is useful as
well. You'll need to learn React Native as part of getting
involved. There's tons of good online tutorials, courses, etc.
### Electron desktop app
@@ -542,7 +542,7 @@ Experts: Anders Kaseorg, Akash Nimare, Abhighyan Khaund.
the open issues and get involved!
**Skills required**: JavaScript experience, Electron experience. You
can learn electron as part of your application!
can learn electron as part of your application!
Good preparation for desktop app projects is to (1) try out the app
and see if you can find bugs or polish problems lacking open issues

2
docs/contributing/summer-with-zulip.md
View File

@@ -225,7 +225,6 @@ materials](https://developers.google.com/open-source/gsoc/resources/manual).
your contributions to the open source world this summer will be something you
can be proud of for the rest of your life.
## What makes a successful summer
Success for the student means a few things, in order of importance:
@@ -247,7 +246,6 @@ Success for the student means a few things, in order of importance:
student has implemented. That section of code should be more readable,
better-tested, and have clearer documentation.
## Extra notes for mentors
- You're personally accountable for your student having a successful summer. If

134
docs/contributing/version-control.md
View File

@@ -11,51 +11,51 @@ helps a lot in preventing bugs.
Commits must be coherent:
- It should pass tests (so test updates needed by a change should be
in the same commit as the original change, not a separate "fix the
tests that were broken by the last commit" commit).
- It should be safe to deploy individually, or explain in detail in
the commit message as to why it isn't (maybe with a [manual] tag).
So implementing a new API endpoint in one commit and then adding the
security checks in a future commit should be avoided -- the security
checks should be there from the beginning.
- Error handling should generally be included along with the code that
might trigger the error.
- TODO comments should be in the commit that introduces the issue or
the functionality with further work required.
- It should pass tests (so test updates needed by a change should be
in the same commit as the original change, not a separate "fix the
tests that were broken by the last commit" commit).
- It should be safe to deploy individually, or explain in detail in
the commit message as to why it isn't (maybe with a [manual] tag).
So implementing a new API endpoint in one commit and then adding the
security checks in a future commit should be avoided -- the security
checks should be there from the beginning.
- Error handling should generally be included along with the code that
might trigger the error.
- TODO comments should be in the commit that introduces the issue or
the functionality with further work required.
Commits should generally be minimal:
- Significant refactorings should be done in a separate commit from
functional changes.
- Moving code from one file to another should be done in a separate
commits from functional changes or even refactoring within a file.
- 2 different refactorings should be done in different commits.
- 2 different features should be done in different commits.
- If you find yourself writing a commit message that reads like a list
of somewhat dissimilar things that you did, you probably should have
just done multiple commits.
- Significant refactorings should be done in a separate commit from
functional changes.
- Moving code from one file to another should be done in a separate
commits from functional changes or even refactoring within a file.
- 2 different refactorings should be done in different commits.
- 2 different features should be done in different commits.
- If you find yourself writing a commit message that reads like a list
of somewhat dissimilar things that you did, you probably should have
just done multiple commits.
When not to be overly minimal:
- For completely new features, you don't necessarily need to split out
new commits for each little subfeature of the new feature. E.g., if
you're writing a new tool from scratch, it's fine to have the
initial tool have plenty of options/features without doing separate
commits for each one. That said, reviewing a 2000-line giant blob of
new code isn't fun, so please be thoughtful about submitting things
in reviewable units.
- Don't bother to split backend commits from frontend commits, even
though the backend can often be coherent on its own.
- For completely new features, you don't necessarily need to split out
new commits for each little subfeature of the new feature. E.g., if
you're writing a new tool from scratch, it's fine to have the
initial tool have plenty of options/features without doing separate
commits for each one. That said, reviewing a 2000-line giant blob of
new code isn't fun, so please be thoughtful about submitting things
in reviewable units.
- Don't bother to split backend commits from frontend commits, even
though the backend can often be coherent on its own.
Other considerations:
- Overly fine commits are easy to squash later, but not vice versa.
So err toward small commits, and the code reviewer can advise on
squashing.
- If a commit you write doesn't pass tests, you should usually fix
that by amending the commit to fix the bug, not writing a new "fix
tests" commit on top of it.
- Overly fine commits are easy to squash later, but not vice versa.
So err toward small commits, and the code reviewer can advise on
squashing.
- If a commit you write doesn't pass tests, you should usually fix
that by amending the commit to fix the bug, not writing a new "fix
tests" commit on top of it.
Zulip expects you to structure the commits in your pull requests to form
a clean history before we will merge them. It's best to write your
@@ -91,6 +91,7 @@ First, check out
of commits with good commit messages.
The first line of the commit message is the **summary**. The summary:
- is written in the imperative (e.g., "Fix ...", "Add ...")
- is kept short (max 76 characters, ideally less), while concisely
explaining what the commit does
@@ -105,7 +106,7 @@ prefix "provision:", using lowercase "**p**". Next, "Improve performance of
install npm." starts with a capital "**I**", uses imperative tense,
and ends with a period.
> *provision: Improve performance of installing npm.*
> _provision: Improve performance of installing npm._
Here are some more positive examples:
@@ -121,16 +122,15 @@ Here are some more positive examples:
> gather_subscriptions: Fix exception handling bad input.
Compare "_gather_subscriptions: Fix exception handling bad input._" with:
Compare "*gather_subscriptions: Fix exception handling bad input.*" with:
- "*gather_subscriptions was broken*", which doesn't explain how
- "_gather_subscriptions was broken_", which doesn't explain how
it was broken (and isn't in the imperative)
- "*Fix exception when given bad input*", in which it's impossible to
- "_Fix exception when given bad input_", in which it's impossible to
tell from the summary what part of the codebase was changed
- "*gather_subscriptions: Fixing exception when given bad input.*",
- "_gather_subscriptions: Fixing exception when given bad input._",
not in the imperative
- "*gather_subscriptions: Fixed exception when given bad input.*",
- "_gather_subscriptions: Fixed exception when given bad input._",
not in the imperative
The summary is followed by a blank line, and then the body of the
@@ -143,29 +143,29 @@ automatically catch common mistakes in the commit message itself.
### Message body:
- The body is written in prose, with full paragraphs; each paragraph should
be separated from the next by a single blank line.
- The body explains:
- why and how the change was made
- any manual testing you did in addition to running the automated tests
- any aspects of the commit that you think are questionable and
you'd like special attention applied to.
- If the commit makes performance improvements, you should generally
include some rough benchmarks showing that it actually improves the
performance.
- When you fix a GitHub issue, [mark that you've fixed the issue in
your commit
message](https://help.github.com/en/articles/closing-issues-via-commit-messages)
so that the issue is automatically closed when your code is merged.
Zulip's preferred style for this is to have the final paragraph of
the commit message read e.g. "Fixes: \#123.".
- Avoid `Partially fixes #1234`; GitHub's regular expressions ignore
the "partially" and close the issue. `Fixes part of #1234` is a good alternative.
- Any paragraph content in the commit message should be line-wrapped
to about 68 characters per line, but no more than 70, so that your
commit message will be reasonably readable in `git log` in a normal
terminal. You may find it helpful to:
- configure Git to use your preferred editor, with the EDITOR
- The body is written in prose, with full paragraphs; each paragraph should
be separated from the next by a single blank line.
- The body explains:
- why and how the change was made
- any manual testing you did in addition to running the automated tests
- any aspects of the commit that you think are questionable and
you'd like special attention applied to.
- If the commit makes performance improvements, you should generally
include some rough benchmarks showing that it actually improves the
performance.
- When you fix a GitHub issue, [mark that you've fixed the issue in
your commit
message](https://help.github.com/en/articles/closing-issues-via-commit-messages)
so that the issue is automatically closed when your code is merged.
Zulip's preferred style for this is to have the final paragraph of
the commit message read e.g. "Fixes: \#123.".
- Avoid `Partially fixes #1234`; GitHub's regular expressions ignore
the "partially" and close the issue. `Fixes part of #1234` is a good alternative.
- Any paragraph content in the commit message should be line-wrapped
to about 68 characters per line, but no more than 70, so that your
commit message will be reasonably readable in `git log` in a normal
terminal. You may find it helpful to:
- configure Git to use your preferred editor, with the EDITOR
environment variable or `git config --global core.editor`, and
- configure the editor to automatically wrap text to 70 or fewer
- configure the editor to automatically wrap text to 70 or fewer
columns per line (all text editors support this).

42
docs/contributing/zulipbot-usage.md
View File

@@ -14,8 +14,8 @@ model GitHub supports).
## Usage
- **Claim an issue** — Comment `@zulipbot claim` on the issue you want
to claim; **@zulipbot** will assign you to the issue and label the issue as
**in progress**.
to claim; **@zulipbot** will assign you to the issue and label the issue as
**in progress**.
- If you're a new contributor, **@zulipbot** will give you read-only
collaborator access to the repository and leave a welcome message on the
@@ -28,8 +28,8 @@ to claim; **@zulipbot** will assign you to the issue and label the issue as
`@zulipbot abandon` to abandon an issue.
- **Label your issues** — Add appropriate labels to issues that you opened by
including `@zulipbot add` in an issue comment or the body of your issue
followed by the desired labels enclosed within double quotes (`""`).
including `@zulipbot add` in an issue comment or the body of your issue
followed by the desired labels enclosed within double quotes (`""`).
- For example, to add the **bug** and **help wanted** labels to your
issue, comment or include `@zulipbot add "bug" "help wanted"` in the
@@ -43,8 +43,8 @@ followed by the desired labels enclosed within double quotes (`""`).
(`""`).
- **Find unclaimed issues** — Use the [GitHub search
feature](https://help.github.com/en/articles/using-search-to-filter-issues-and-pull-requests)
to find unclaimed issues by adding one of the following filters to your search:
feature](https://help.github.com/en/articles/using-search-to-filter-issues-and-pull-requests)
to find unclaimed issues by adding one of the following filters to your search:
- `-label: "in progress"` (excludes issues labeled with the **in progress** label)
@@ -54,19 +54,19 @@ to find unclaimed issues by adding one of the following filters to your search:
already been claimed.
- **Collaborate in area label teams** — Receive notifications on
issues and pull requests within your fields of expertise on the
[Zulip server repository](https://github.com/zulip/zulip) by joining
the Zulip server
[area label teams](https://github.com/orgs/zulip/teams?utf8=✓&query=Server)
(Note: this link only works for members of the Zulip organization;
we'll happily add you if you're interested). These teams correspond
to the repository's
[area labels](https://github.com/zulip/zulip/labels), although some
teams are associated with multiple labels; for example, the **area:
message-editing** and **area: message view** labels are both related
to the
[Server message view](https://github.com/orgs/zulip/teams/server-message-view)
team. Feel free to join as many area label teams as as you'd like!
issues and pull requests within your fields of expertise on the
[Zulip server repository](https://github.com/zulip/zulip) by joining
the Zulip server
[area label teams](https://github.com/orgs/zulip/teams?utf8=✓&query=Server)
(Note: this link only works for members of the Zulip organization;
we'll happily add you if you're interested). These teams correspond
to the repository's
[area labels](https://github.com/zulip/zulip/labels), although some
teams are associated with multiple labels; for example, the **area:
message-editing** and **area: message view** labels are both related
to the
[Server message view](https://github.com/orgs/zulip/teams/server-message-view)
team. Feel free to join as many area label teams as as you'd like!
After your request to join an area label team is approved, you'll receive
notifications for any issues labeled with the team's corresponding area
@@ -74,8 +74,8 @@ team. Feel free to join as many area label teams as as you'd like!
team's area label.
- **Track inactive claimed issues** — If a claimed issue has not been updated
for a week, **@zulipbot** will post a comment on the inactive issue to ask the
assignee(s) if they are still working on the issue.
for a week, **@zulipbot** will post a comment on the inactive issue to ask the
assignee(s) if they are still working on the issue.
If you see this comment on an issue you claimed, you should post a comment
on the issue to notify **@zulipbot** that you're still working on it.

53
docs/development/authentication.md
View File

@@ -46,9 +46,9 @@ details worth understanding:
### Google
- Visit [the Google developer
console](https://console.developers.google.com) and navigate to "APIs
& services" > "Credentials". Create a "Project", which will correspond
to your dev environment.
console](https://console.developers.google.com) and navigate to "APIs
& services" > "Credentials". Create a "Project", which will correspond
to your dev environment.
- Navigate to "APIs & services" > "Library", and find the "Identity
Toolkit API". Choose "Enable".
@@ -96,12 +96,12 @@ to your dev environment.
- [Create a Sign in with Apple private key](https://help.apple.com/developer-account/?lang=en#/dev77c875b7e)
- In `dev-secrets.conf`, set
- `social_auth_apple_services_id` to your
"Services ID" (eg. com.application.your).
- `social_auth_apple_app_id` to "App ID" or "Bundle ID".
This is only required if you are testing Apple auth on iOS.
- `social_auth_apple_key` to your "Key ID".
- `social_auth_apple_team` to your "Team ID".
- `social_auth_apple_services_id` to your
"Services ID" (eg. com.application.your).
- `social_auth_apple_app_id` to "App ID" or "Bundle ID".
This is only required if you are testing Apple auth on iOS.
- `social_auth_apple_key` to your "Key ID".
- `social_auth_apple_team` to your "Team ID".
- Put the private key file you got from apple at the path
`zproject/dev_apple.key`.
@@ -111,21 +111,21 @@ to your dev environment.
- Set up SAML authentication by following
[Okta's documentation](https://developer.okta.com/docs/guides/saml-application-setup/overview/).
Specify:
- `http://localhost:9991/complete/saml/` for the "Single sign on URL"`.
- `http://localhost:9991` for the "Audience URI (SP Entity ID)".
- Skip "Default RelayState".
- Skip "Name ID format".
- Set 'Email` for "Application username format".
- Provide "Attribute statements" of `email` to `user.email`,
`first_name` to `user.firstName`, and `last_name` to `user.lastName`.
- `http://localhost:9991/complete/saml/` for the "Single sign on URL"`.
- `http://localhost:9991` for the "Audience URI (SP Entity ID)".
- Skip "Default RelayState".
- Skip "Name ID format".
- Set 'Email` for "Application username format".
- Provide "Attribute statements" of `email` to `user.email`,
`first_name` to `user.firstName`, and `last_name` to `user.lastName`.
- Assign at least one account in the "Assignments" tab. You'll use it for
signing up / logging in to Zulip.
- Visit the big "Setup instructions" button on the "Sign on" tab.
- Edit `zproject/dev-secrets.conf` to add the two values provided:
- Set `saml_url = http...` from "Identity Provider Single Sign-On
URL".
- Set `saml_entity_id = http://...` from "Identity Provider Issuer".
- Download the certificate and put it at the path `zproject/dev_saml.cert`.
- Set `saml_url = http...` from "Identity Provider Single Sign-On
URL".
- Set `saml_entity_id = http://...` from "Identity Provider Issuer".
- Download the certificate and put it at the path `zproject/dev_saml.cert`.
- Now you should have working SAML authentication!
- You can sign up to the target realm with the account that you've "assigned"
in the previous steps (if the account's email address is allowed in the realm,
@@ -159,12 +159,13 @@ development environment as well, so that you can go through all the
actual flows for LDAP configuration.
- To enable fakeldap, set `FAKE_LDAP_MODE` in
`zproject/dev_settings.py` to one of the following options. For more
information on these modes, refer to
[our production docs](../production/authentication-methods.html#ldap-including-active-directory):
`zproject/dev_settings.py` to one of the following options. For more
information on these modes, refer to
[our production docs](../production/authentication-methods.html#ldap-including-active-directory):
- `a`: If users' email addresses are in LDAP and used as username.
- `b`: If LDAP only has usernames but email addresses are of the form
username@example.com
username@example.com
- `c`: If LDAP usernames are completely unrelated to email addresses.
- To disable fakeldap, set `FAKE_LDAP_MODE` back to `None`.
@@ -174,8 +175,8 @@ information on these modes, refer to
`ldapuser1`).
- `FAKE_LDAP_NUM_USERS` in `zproject/dev_settings.py` can be used to
specify the number of LDAP users to be added. The default value for
the number of LDAP users is 8.
specify the number of LDAP users to be added. The default value for
the number of LDAP users is 8.
### Testing avatar and custom profile field synchronization

67
docs/development/remote.md
View File

@@ -19,7 +19,7 @@ The best way to connect to your server is using the command line tool `ssh`.
- On macOS and Linux/UNIX, `ssh` is a part of Terminal.
- On Windows, `ssh` comes with [Bash for Git][git-bash].
Open *Terminal* or *Bash for Git*, and connect with the following:
Open _Terminal_ or _Bash for Git_, and connect with the following:
```console
$ ssh username@host
@@ -37,14 +37,15 @@ to the next section.
You can create a new user with sudo privileges by running the
following commands as root:
- You can create a `zulipdev` user by running the command
`adduser zulipdev`. Run through the prompts to assign a password and
user information. (You can pick any username you like for this user
account.)
`adduser zulipdev`. Run through the prompts to assign a password and
user information. (You can pick any username you like for this user
account.)
- You can add the user to the sudo group by running the command
`usermod -aG sudo zulipdev`.
`usermod -aG sudo zulipdev`.
- Finally, you can switch to the user by running the command
`su - zulipdev` (or just log in to that user using `ssh`).
`su - zulipdev` (or just log in to that user using `ssh`).
## Setting up the development environment
@@ -149,7 +150,7 @@ guide][rtd-git-guide]. In brief, the steps are as follows.
On your **local computer**:
1. Open *Terminal* (macOS/Linux) or *Git for BASH*.
1. Open _Terminal_ (macOS/Linux) or _Git for BASH_.
2. Change directory to where you cloned Zulip (e.g. `cd zulip`).
3. Use `git add` and `git commit` to stage and commit your changes (if you
haven't already).
@@ -160,7 +161,7 @@ Be sure to replace `branchname` with the name of your actual feature branch.
Once `git push` has completed successfully, you are ready to fetch the commits
from your remote development instance:
1. In *Terminal* or *Git BASH*, connect to your remote development
1. In _Terminal_ or _Git BASH_, connect to your remote development
instance with `ssh user@host`.
2. Change to the zulip directory (e.g., `cd zulip`).
3. Fetch new commits from GitHub with `git fetch origin`.
@@ -193,12 +194,13 @@ Similar packages/extensions exist for other popular code editors as
well; contributions of precise documentation for them are welcome!
- [VSCode Remote - SSH][vscode-remote-ssh]: Lets you use Visual Studio
Code against a remote repository with a similar user experience to
developing locally.
Code against a remote repository with a similar user experience to
developing locally.
[vscode-remote-ssh]: https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-ssh
- [rmate](https://github.com/textmate/rmate) for TextMate + VS Code:
1. Install the extension
[Remote VSCode](https://marketplace.visualstudio.com/items?itemName=rafaelmaiolla.remote-vscode).
2. On your remote machine, run:
@@ -229,11 +231,11 @@ Two editors often available by default on Linux systems are:
- **Nano**: A very simple, beginner-friendly editor. However, it lacks a lot of
features useful for programming, such as syntax highlighting, so we only
recommended it for quick edits to things like configuration files. Launch by
running command `nano <filename>`. Exit by pressing *Ctrl-X*.
running command `nano <filename>`. Exit by pressing _Ctrl-X_.
- **[Vim](https://www.vim.org/)**: A very powerful editor that can take a while
to learn. Launch by running `vim <filename>`. Quit Vim by pressing *Esc*,
typing `:q`, and then pressing *Enter*. Vim comes with a program to learn it
to learn. Launch by running `vim <filename>`. Quit Vim by pressing _Esc_,
typing `:q`, and then pressing _Enter_. Vim comes with a program to learn it
called `vimtutor` (just run that command to start it).
Other options include:
@@ -250,12 +252,12 @@ started working quickly, we recommend web-based IDE
To set up Codeanywhere for Zulip:
1. Create a [Codeanywhere][codeanywhere] account and log in.
2. Create a new **SFTP-SSH** project. Use *Public key* for authentication.
2. Create a new **SFTP-SSH** project. Use _Public key_ for authentication.
3. Click **GET YOUR PUBLIC KEY** to get the new public key that
Codeanywhere generates when you create a new project. Add this public key to
`~/.ssh/authorized_keys` on your remote development instance.
4. Once you've added the new public key to your remote development instance, click
*CONNECT*.
_CONNECT_.
Now your workspace should look similar this:
![Codeanywhere workspace][img-ca-workspace]
@@ -290,26 +292,27 @@ that the user is `zulipdev`; edit accordingly if the situation is
different.
1. First, get an SSL certificate; you can use
[our certbot wrapper script used for production](../production/ssl-certificates.html#certbot-recommended)
by running the following commands as root:
```bash
# apt install -y crudini
mkdir -p /var/lib/zulip/certbot-webroot/
# if nginx running this will fail and you need to run `service nginx stop`
/home/zulipdev/zulip/scripts/setup/setup-certbot \
hostname.example.com --no-zulip-conf \
--email=username@example.com --method=standalone
```
[our certbot wrapper script used for production](../production/ssl-certificates.html#certbot-recommended)
by running the following commands as root:
```bash
# apt install -y crudini
mkdir -p /var/lib/zulip/certbot-webroot/
# if nginx running this will fail and you need to run `service nginx stop`
/home/zulipdev/zulip/scripts/setup/setup-certbot \
hostname.example.com --no-zulip-conf \
--email=username@example.com --method=standalone
```
1. Install nginx configuration:
```bash
apt install -y nginx-full
cp -a /home/zulipdev/zulip/tools/droplets/zulipdev /etc/nginx/sites-available/
ln -nsf /etc/nginx/sites-available/zulipdev /etc/nginx/sites-enabled/
nginx -t # Verifies your nginx configuration
service nginx reload # Actually enabled your nginx configuration
```
```bash
apt install -y nginx-full
cp -a /home/zulipdev/zulip/tools/droplets/zulipdev /etc/nginx/sites-available/
ln -nsf /etc/nginx/sites-available/zulipdev /etc/nginx/sites-enabled/
nginx -t # Verifies your nginx configuration
service nginx reload # Actually enabled your nginx configuration
```
1. Edit `zproject/dev_settings.py` to set
`EXTERNAL_URI_SCHEME = "https://"`, so that URLs served by the

3
docs/development/setup-advanced.md
View File

@@ -64,7 +64,7 @@ the [WSL 2](https://docs.microsoft.com/en-us/windows/wsl/wsl2-about)
installation method described here.
1. Install WSL 2 by following the instructions provided by Microsoft
[here](https://docs.microsoft.com/en-us/windows/wsl/wsl2-install).
[here](https://docs.microsoft.com/en-us/windows/wsl/wsl2-install).
1. Install the `Ubuntu 18.04` Linux distribution from the Microsoft
Store.
@@ -92,6 +92,7 @@ installation method described here.
1. Make sure you are inside the WSL disk and not in a Windows mounted disk.
You will run into permission issues if you run `provision` from `zulip`
in a Windows mounted disk.
```bash
cd ~ # or cd /home/USERNAME
```

23
docs/development/setup-vagrant.md
View File

@@ -10,6 +10,7 @@ or a Linux container (for Ubuntu) inside which the Zulip server and
all related services will run.
Contents:
- [Requirements](#requirements)
- [Step 0: Set up Git & GitHub](#step-0-set-up-git-github)
- [Step 1: Install prerequisites](#step-1-install-prerequisites)
@@ -159,11 +160,11 @@ Debian](https://docs.docker.com/install/linux/docker-ce/debian/).
We recommend using `WSL 2 for Windows development <../development/setup-advanced.html#installing-directly-on-windows-10-with-wsl-2>`_.
```
1. Install [Git for Windows][git-bash], which installs *Git BASH*.
1. Install [Git for Windows][git-bash], which installs _Git BASH_.
2. Install [VirtualBox][vbox-dl] (latest).
3. Install [Vagrant][vagrant-dl] (latest).
(Note: While *Git BASH* is recommended, you may also use [Cygwin][cygwin-dl].
(Note: While _Git BASH_ is recommended, you may also use [Cygwin][cygwin-dl].
If you do, make sure to **install default required packages** along with
**git**, **curl**, **openssh**, and **rsync** binaries.)
@@ -363,6 +364,7 @@ Quit the server with CTRL-C.
2016-05-04 18:20:40,722 INFO Tornado 95.5% busy over the past 0.0 seconds
Performing system checks...
```
And ending with something similar to:
```console
@@ -494,6 +496,7 @@ logout
Connection to 127.0.0.1 closed.
christie@win10 ~/zulip
```
Now you can suspend the development environment:
```console
@@ -537,8 +540,8 @@ Next, read the following to learn more about developing for Zulip:
- [Git & GitHub guide][rtd-git-guide]
- [Using the development environment][rtd-using-dev-env]
- [Testing][rtd-testing] (and [Configuring CI][ci] to
run the full test suite against any branches you push to your fork,
which can help you optimize your development workflow).
run the full test suite against any branches you push to your fork,
which can help you optimize your development workflow).
### Troubleshooting and common errors
@@ -590,6 +593,7 @@ shell and run `vagrant ssh` again to get the virtualenv setup properly.
#### Vagrant was unable to mount VirtualBox shared folders
For the following error:
```console
Vagrant was unable to mount VirtualBox shared folders. This is usually
because the filesystem "vboxsf" is not available. This filesystem is
@@ -810,6 +814,7 @@ proxy to access the Internet and haven't [configured the development
environment to use it](#specifying-a-proxy).
Once you've provisioned successfully, you'll get output like this:
```console
Zulip development environment setup succeeded!
(zulip-py3-venv) vagrant@vagrant-base-trusty-amd64:~/zulip$
@@ -830,10 +835,10 @@ Likely causes are:
1. Networking issues
2. Insufficient RAM. Check whether you've allotted at least two
gigabytes of RAM, which is the minimum Zulip
[requires](../development/setup-vagrant.html#requirements). If
not, go to your VM settings and increase the RAM, then restart
the VM.
gigabytes of RAM, which is the minimum Zulip
[requires](../development/setup-vagrant.html#requirements). If
not, go to your VM settings and increase the RAM, then restart
the VM.
##### yarn install warnings
@@ -904,7 +909,7 @@ vagrant ssh -- 'modinfo -F version vboxsf'
The bug has not been fixed upstream as of this writing, but you may be
able to work around it by downgrading VirtualBox Guest Additions to
6.0.4. To do this, create a `~/.zulip-vagrant-config` file and add
6.0.4. To do this, create a `~/.zulip-vagrant-config` file and add
this line:
```text

15
docs/development/test-install.md
View File

@@ -1,12 +1,12 @@
# Testing the installer
Zulip's install process is tested as part of [its continuous
integrations suite][CI], but that only tests the most common
integrations suite][ci], but that only tests the most common
configurations; when making changes to more complicated [installation
options][installer-docs], Zulip provides tooling to repeatedly test
the installation process in a clean environment each time.
[CI]: https://github.com/zulip/zulip/actions/workflows/production-suite.yml?query=branch%3Amain
[ci]: https://github.com/zulip/zulip/actions/workflows/production-suite.yml?query=branch%3Amain
[installer-docs]: ../production/install.md
## Configuring
@@ -17,6 +17,7 @@ RAM, in order to accommodate the VMs and the steps which build the
release assets.
To begin, install the LXC toolchain:
```bash
sudo apt-get install lxc lxc-utils
```
@@ -32,6 +33,7 @@ You only need to do this step once per time you work on a set of
changes, to refresh the package that the installer uses. The installer
doesn't work cleanly out of a source checkout; it wants a release
checkout, so we build a tarball of one of those first:
```bash
./tools/build-release-tarball test-installer
```
@@ -46,6 +48,7 @@ directory. The test installer needs the release directory to be named
`zulip-server`, so we rename it and move it appropriately. In the
first line, you'll need to substitute the actual path that you got for
the tarball, above:
```bash
tar xzf /tmp/tmp.fepqqNBWxp/zulip-server-test-installer.tar.gz
mkdir zulip-test-installer
@@ -65,6 +68,7 @@ into the installer.
For example, to test an install onto Ubuntu 20.04 "Focal", we might
call:
```bash
sudo ./tools/test-install/install \
-r focal \
@@ -82,6 +86,7 @@ take a while.
Regardless of if the install succeeds or fails, it will stay running
so you can inspect it. You can see all of the containers which are
running, and their randomly-generated names, by running:
```bash
sudo lxc-ls -f
```
@@ -90,6 +95,7 @@ sudo lxc-ls -f
After using `lxc-ls` to list containers, you can choose one of them
and connect to its terminal:
```bash
sudo lxc-attach --clear-env -n zulip-install-focal-PUvff
```
@@ -98,23 +104,24 @@ sudo lxc-attach --clear-env -n zulip-install-focal-PUvff
To destroy all containers (but leave the base containers, which speed
up the initial install):
```bash
sudo ./tools/test-install/destroy-all -f
```
To destroy just one container:
```bash
sudo lxc-destroy -f -n zulip-install-focal-PUvff
```
### Iterating on the installer
Iterate on the installer by making changes to your source tree,
copying them into the release directory, and re-running the installer,
which will start up a new container. Here, we update just the
`scripts` and `puppet` directories of the release directory:
```bash
rsync -az scripts puppet zulip-test-installer/zulip-server/

11
docs/development/using.md
View File

@@ -1,5 +1,4 @@
Using the development environment
=================================
# Using the development environment
This page describes the basic edit/refresh workflows for working with
the Zulip development environment. Generally, the development
@@ -40,12 +39,12 @@ the development environment][authentication-dev-server].
- For changes that don't affect the database model, the Zulip
development environment will automatically detect changes and
restart:
- The main Django/Tornado server processes are run on top of
- The main Django/Tornado server processes are run on top of
Django's [manage.py runserver][django-runserver], which will
automatically restart them when you save changes to Python code
they use. You can watch this happen in the `run-dev.py` console
to make sure the backend has reloaded.
- The Python queue workers will also automatically restart when you
- The Python queue workers will also automatically restart when you
save changes, as long as they haven't crashed (which can happen if
they reloaded into a version with a syntax error).
- If you change the database schema (`zerver/models.py`), you'll need
@@ -69,9 +68,9 @@ the development environment][authentication-dev-server].
- By default, the development server homepage just shows a list of the
users that exist on the server and you can log in as any of them by
just clicking on a user.
- This setup saves time for the common case where you want to test
- This setup saves time for the common case where you want to test
something other than the login process.
- You can test the login or registration process by clicking the
- You can test the login or registration process by clicking the
links for the normal login page.
- Most changes will take effect automatically. Details:
- If you change CSS files, your changes will appear immediately via

7
docs/documentation/api.md
View File

@@ -124,7 +124,7 @@ For the Python examples, you'll write the example in
automatically in Zulip's automated test suite. The code there will
look something like this:
``` python
```python
@openapi_test_function('/messages/render:post')
def render_message(client: Client) -> None:
# {code_example|start}
@@ -215,10 +215,11 @@ above.
declared using `REQ`.
You can check your formatting using these helpful tools.
- `tools/check-openapi` will verify the syntax of `zerver/openapi/zulip.yaml`.
- `tools/test-backend zerver/tests/test_openapi.py`; this test compares
your documentation against the code and can find many common
mistakes in how arguments are declared.
your documentation against the code and can find many common
mistakes in how arguments are declared.
- `test-backend`: The full Zulip backend test suite will fail if
any actual API responses generated by the tests don't match your
defined OpenAPI schema. Use `test-backend --rerun` for a fast

23
docs/documentation/integrations.md
View File

@@ -84,12 +84,12 @@ Here are a few common macros used to document Zulip's integrations:
This macro is usually used right after `{!create-stream!}`. For an example
rendering, see **Step 2** of [the docs for Zulip's GitHub integration][github-integration].
**Note:** If special configuration is
required to set up the URL and you can't use this macro, be sure to use the
`{{ api_url }}` template variable, so that your integration
documentation will provide the correct URL for whatever server it is
deployed on. If special configuration is required to set the `SITE`
variable, you should document that too.
**Note:** If special configuration is
required to set up the URL and you can't use this macro, be sure to use the
`{{ api_url }}` template variable, so that your integration
documentation will provide the correct URL for whatever server it is
deployed on. If special configuration is required to set the `SITE`
variable, you should document that too.
- `{!append-stream-name.md!}` macro - Recommends appending `&stream=stream_name`
to a URL in cases where supplying a stream name in the URL is optional.
@@ -136,12 +136,12 @@ Here are a few common macros used to document Zulip's integrations:
- `{!webhook-url-with-bot-email.md!}` - Used in certain non-webhook integrations
to generate URLs of the form:
```text
https://bot_email:bot_api_key@yourZulipDomain.zulipchat.com/api/v1/external/beanstalk
```
```text
https://bot_email:bot_api_key@yourZulipDomain.zulipchat.com/api/v1/external/beanstalk
```
For an example rendering, see
[Zulip's Beanstalk integration](https://zulip.com/integrations/doc/beanstalk).
For an example rendering, see
[Zulip's Beanstalk integration](https://zulip.com/integrations/doc/beanstalk).
[github-integration]: https://zulip.com/integrations/doc/github
[codebase]: https://zulip.com/integrations/doc/codebase
@@ -189,7 +189,6 @@ concrete guidelines.
- Follow the organization and wording of existing docs as much as possible.
### Guidelines for specific steps
Most doc files should start with a generic sentence about the

1
docs/documentation/openapi.md
View File

@@ -40,6 +40,7 @@ types of authentication, and configure other settings. Once defined,
information in this section rarely changes.
For example, the `swagger` and `info` objects look like this:
```yaml
# Basic Swagger UI info
openapi: 3.0.1

20
docs/documentation/overview.md
View File

@@ -18,11 +18,11 @@ Zulip has three major documentation systems:
several hundred pages written using this system. There are 3
branches of this documentation:
- User documentation (with a target audience of individual Zulip
users),
users),
- Integrations documentation (with a target audience of IT folks
setting up integrations), and
setting up integrations), and
- API documentation (with a target audience of developers writing
code to extend Zulip).
code to extend Zulip).
These three systems are documented in detail.
@@ -124,21 +124,21 @@ Zulip has several automated test suites that we run in CI and
recommend running locally when making significant edits:
- `tools/lint` catches a number of common mistakes, and we highly
recommend
[using our linter pre-commit hook](../git/zulip-tools.html#set-up-git-repo-script).
See the [main linter doc](../testing/linters.md) for more details.
recommend
[using our linter pre-commit hook](../git/zulip-tools.html#set-up-git-repo-script).
See the [main linter doc](../testing/linters.md) for more details.
- The ReadTheDocs docs are built and the links tested by
`tools/test-documentation`, which runs `build-docs` and then checks
all the links.
`tools/test-documentation`, which runs `build-docs` and then checks
all the links.
There's an exclude list for the link testing at this horrible path:
`tools/documentation_crawler/documentation_crawler/spiders/common/spiders.py`,
which is relevant for flaky links.
- The API docs are tested by `tools/test-api`, which does some basic
payload verification. Note that this test does not check for broken
links (those are checked by `test-help-documentation`).
payload verification. Note that this test does not check for broken
links (those are checked by `test-help-documentation`).
- `tools/test-help-documentation` checks `/help/`, `/api/`,
`/integrations/`, and the core website ("portico") documentation for

47
docs/documentation/user.md
View File

@@ -8,6 +8,7 @@ There are two types of documents: articles about specific features, and a
handful of longer guides.
The feature articles serve a few different purposes:
- Feature discovery, for someone browsing the `/help` page, and looking at
the set of titles.
- Public documentation of our featureset, for someone googling "can zulip do .."
@@ -29,7 +30,7 @@ ReadTheDocs, since Zulip supports running a server completely disconnected
from the Internet, and we'd like the documentation to be available in that
environment.
The source for this user documentation is the Markdown files under
The source for this user documentation is the Markdown files under
`templates/zerver/help/` in the
[main Zulip server repository](https://github.com/zulip/zulip). The file
`foo.md` is automatically rendered by the `render_markdown_path` function in
@@ -99,15 +100,15 @@ Zulip's Markdown processor allows you to include several special features in
your documentation to help improve its readability:
- Since raw HTML is supported in Markdown, you can include arbitrary
HTML/CSS in your documentation as needed.
HTML/CSS in your documentation as needed.
- Code blocks allow you to highlight syntax, similar to Zulip's own Markdown.
- Anchor tags can be used to link to headers in other documents.
- [Images](#images) of Zulip UI can be added to documentation.
- Inline [icons](#icons) used to refer to features in the Zulip UI.
- You can utilize [macros](#macros) to limit repeated content in the
documentation.
documentation.
- You can create special highlight warning blocks using
[tips and warnings](#tips-and-warnings).
[tips and warnings](#tips-and-warnings).
- You can create tabs using [Markdown tab switcher](#tab-switcher).
### Images
@@ -127,7 +128,7 @@ instructions for something simple look long and complicated.
When taking screenshots, the image should never include the whole
Zulip browser window in a screenshot; instead, it should only show
relevant parts of the app. In addition, the screenshot should always
come *after* the text that describes it, never before.
come _after_ the text that describes it, never before.
Images are often a part of a numbered step and must be indented four
spaces to be formatted correctly.
@@ -142,39 +143,39 @@ icons from [FontAwesome](https://fontawesome.com/v4.7.0/) (version 4.7.0) which
make use of `fa` as a base class.
- cog (<i class="fa fa-cog"></i>) icon —
`cog (<i class="fa fa-cog"></i>) icon`
`cog (<i class="fa fa-cog"></i>) icon`
- down chevron (<i class="fa fa-chevron-down"></i>) icon —
`down chevron (<i class="fa fa-chevron-down"></i>) icon`
`down chevron (<i class="fa fa-chevron-down"></i>) icon`
- eye (<i class="fa fa-eye"></i>) icon —
`eye (<i class="fa fa-eye"></i>) icon`
`eye (<i class="fa fa-eye"></i>) icon`
- file (<i class="fa fa-file-code-o"></i>) icon —
`file (<i class="fa fa-file-code-o"></i>) icon`
`file (<i class="fa fa-file-code-o"></i>) icon`
- filled star (<i class="fa fa-star"></i>) icon —
`filled star (<i class="fa fa-star"></i>) icon`
`filled star (<i class="fa fa-star"></i>) icon`
- formatting (<i class="fa fa-font"></i>) icon —
`formatting (<i class="fa fa-font"></i>) icon`
`formatting (<i class="fa fa-font"></i>) icon`
- menu (<i class="fa fa-bars"></i>) icon —
`menu (<i class="fa fa-bars"></i>) icon`
`menu (<i class="fa fa-bars"></i>) icon`
- overflow ( <i class="fa fa-ellipsis-v"></i> ) icon —
`overflow ( <i class="fa fa-ellipsis-v"></i> ) icon`
`overflow ( <i class="fa fa-ellipsis-v"></i> ) icon`
- paperclip (<i class="fa fa-paperclip"></i>) icon —
`paperclip (<i class="fa fa-paperclip"></i>) icon`
`paperclip (<i class="fa fa-paperclip"></i>) icon`
- pencil (<i class="fa fa-pencil"></i>) icon —
`pencil (<i class="fa fa-pencil"></i>) icon`
`pencil (<i class="fa fa-pencil"></i>) icon`
- pencil and paper (<i class="fa fa-pencil-square-o"></i>) icon —
`pencil and paper (<i class="fa fa-pencil-square-o"></i>) icon`
`pencil and paper (<i class="fa fa-pencil-square-o"></i>) icon`
- plus (<i class="fa fa-plus"></i>) icon —
`plus (<i class="fa fa-plus"></i>) icon`
`plus (<i class="fa fa-plus"></i>) icon`
- smiley face (<i class="fa fa-smile-o"></i>) icon —
`smiley face (<i class="fa fa-smile-o"></i>) icon`
`smiley face (<i class="fa fa-smile-o"></i>) icon`
- star (<i class="fa fa-star-o"></i>) icon —
`star (<i class="fa fa-star-o"></i>) icon`
`star (<i class="fa fa-star-o"></i>) icon`
- trash (<i class="fa fa-trash-o"></i>) icon —
`trash (<i class="fa fa-trash-o"></i>) icon`
`trash (<i class="fa fa-trash-o"></i>) icon`
- video-camera (<i class="fa fa-video-camera"></i>) icon —
`video-camera (<i class="fa fa-video-camera"></i>) icon`
`video-camera (<i class="fa fa-video-camera"></i>) icon`
- x (<i class="fa fa-times"></i>) icon —
`x (<i class="fa fa-times"></i>) icon`
`x (<i class="fa fa-times"></i>) icon`
### Macros
@@ -230,7 +231,7 @@ All tips/warnings should appear inside tip/warning blocks. There
should be only one tip/warning inside each block, and they usually
should be formatted as a continuation of a numbered step.
### Tab switcher
### Tab switcher
Our Markdown processor supports easily creating a tab switcher widget
design to easily show the instructions for different

138
docs/git/cheat-sheet.md
View File

@@ -5,111 +5,111 @@ See also [fixing commits][fix-commit]
## Common commands
- add
- `git add foo.py`
- `git add foo.py`
- checkout
- `git checkout -b new-branch-name`
- `git checkout main`
- `git checkout old-branch-name`
- `git checkout -b new-branch-name`
- `git checkout main`
- `git checkout old-branch-name`
- commit
- `git commit -m "topic: Commit message title."`
- `git commit --amend`: Modify the previous commit.
- `git commit -m "topic: Commit message title."`
- `git commit --amend`: Modify the previous commit.
- config
- `git config --global core.editor nano`
- `git config --global core.symlinks true`
- `git config --global core.editor nano`
- `git config --global core.symlinks true`
- diff
- `git diff`
- `git diff --cached`
- `git diff HEAD~2..`
- `git diff`
- `git diff --cached`
- `git diff HEAD~2..`
- fetch
- `git fetch origin`
- `git fetch upstream`
- `git fetch origin`
- `git fetch upstream`
- grep
- `git grep update_unread_counts`
- `git grep update_unread_counts`
- log
- `git log`
- `git log`
- pull
- `git pull --rebase`: **Use this**. Zulip uses a [rebase oriented workflow][git-overview].
- `git pull` (with no options): Will either create a merge commit
(which you don't want) or do the same thing as `git pull --rebase`,
depending on [whether you've configured Git properly][git-config-clone]
- `git pull --rebase`: **Use this**. Zulip uses a [rebase oriented workflow][git-overview].
- `git pull` (with no options): Will either create a merge commit
(which you don't want) or do the same thing as `git pull --rebase`,
depending on [whether you've configured Git properly][git-config-clone]
- push
- `git push origin +branch-name`
- `git push origin +branch-name`
- rebase
- `git rebase -i HEAD~3`
- `git rebase -i main`
- `git rebase upstream/main`
- `git rebase -i HEAD~3`
- `git rebase -i main`
- `git rebase upstream/main`
- reflog
- `git reflog | head -10`
- `git reflog | head -10`
- remote
- `git remote -v`
- `git remote -v`
- reset
- `git reset HEAD~2`
- `git reset HEAD~2`
- rm
- `git rm oops.txt`
- `git rm oops.txt`
- show
- `git show HEAD`
- `git show HEAD~~~`
- `git show main`
- `git show HEAD`
- `git show HEAD~~~`
- `git show main`
- status
- `git status`
- `git status`
## Detailed cheat sheet
- add
- `git add foo.py`: add `foo.py` to the staging area
- `git add foo.py bar.py`: add `foo.py` AND `bar.py` to the staging area
- `git add -u`: Adds all tracked files to the staging area.
- `git add foo.py`: add `foo.py` to the staging area
- `git add foo.py bar.py`: add `foo.py` AND `bar.py` to the staging area
- `git add -u`: Adds all tracked files to the staging area.
- checkout
- `git checkout -b new-branch-name`: create branch `new-branch-name` and switch to/check out that new branch
- `git checkout main`: switch to your `main` branch
- `git checkout old-branch-name`: switch to an existing branch `old-branch-name`
- `git checkout -b new-branch-name`: create branch `new-branch-name` and switch to/check out that new branch
- `git checkout main`: switch to your `main` branch
- `git checkout old-branch-name`: switch to an existing branch `old-branch-name`
- commit
- `git commit -m "commit message"`: It is recommended to type a
multiline commit message, however.
- `git commit`: Opens your default text editor to write a commit message.
- `git commit --amend`: changing the last commit message. Read more [here][fix-commit]
- `git commit -m "commit message"`: It is recommended to type a
multiline commit message, however.
- `git commit`: Opens your default text editor to write a commit message.
- `git commit --amend`: changing the last commit message. Read more [here][fix-commit]
- config
- `git config --global core.editor nano`: set core editor to `nano` (you can set this to `vim` or others)
- `git config --global core.symlinks true`: allow symbolic links
- `git config --global core.editor nano`: set core editor to `nano` (you can set this to `vim` or others)
- `git config --global core.symlinks true`: allow symbolic links
- diff
- `git diff`: display the changes you have made to all files
- `git diff --cached`: display the changes you have made to staged files
- `git diff HEAD~2..`: display the 2 most recent changes you have made to files
- `git diff`: display the changes you have made to all files
- `git diff --cached`: display the changes you have made to staged files
- `git diff HEAD~2..`: display the 2 most recent changes you have made to files
- fetch
- `git fetch origin`: fetch origin repository
- `git fetch upstream`: fetch upstream repository
- `git fetch origin`: fetch origin repository
- `git fetch upstream`: fetch upstream repository
- grep
- `git grep update_unread_counts static/js`: Search our JS for references to update_unread_counts.
- `git grep update_unread_counts static/js`: Search our JS for references to update_unread_counts.
- log
- `git log`: show commit logs
- `git log --oneline | head`: To quickly see the latest ten commits on a branch.
- `git log`: show commit logs
- `git log --oneline | head`: To quickly see the latest ten commits on a branch.
- pull
- `git pull --rebase`: rebase your changes on top of `main`.
- `git pull` (with no options): Will either create a merge commit
(which you don't want) or do the same thing as `git pull --rebase`,
depending on [whether you've configured Git properly][git-config-clone]
- `git pull --rebase`: rebase your changes on top of `main`.
- `git pull` (with no options): Will either create a merge commit
(which you don't want) or do the same thing as `git pull --rebase`,
depending on [whether you've configured Git properly][git-config-clone]
- push
- `git push origin branch-name`: push you commits to the origin repository *only if* there are no conflicts.
Use this when collaborating with others to prevent overwriting their work.
- `git push origin +branch-name`: force push your commits to your origin repository.
- `git push origin branch-name`: push you commits to the origin repository _only if_ there are no conflicts.
Use this when collaborating with others to prevent overwriting their work.
- `git push origin +branch-name`: force push your commits to your origin repository.
- rebase
- `git rebase -i HEAD~3`: interactive rebasing current branch with first three items on HEAD
- `git rebase -i main`: interactive rebasing current branch with `main` branch
- `git rebase upstream/main`: rebasing current branch with `main` branch from upstream repository
- `git rebase -i HEAD~3`: interactive rebasing current branch with first three items on HEAD
- `git rebase -i main`: interactive rebasing current branch with `main` branch
- `git rebase upstream/main`: rebasing current branch with `main` branch from upstream repository
- reflog
- `git reflog | head -10`: manage reference logs for the past 10 commits
- `git reflog | head -10`: manage reference logs for the past 10 commits
- remote
- `git remote -v`: display your origin and upstream repositories
- `git remote -v`: display your origin and upstream repositories
- reset
- `git reset HEAD~2`: reset two most recent commits
- `git reset HEAD~2`: reset two most recent commits
- rm
- `git rm oops.txt`: remove `oops.txt`
- `git rm oops.txt`: remove `oops.txt`
- show
- `git show HEAD`: display most recent commit
- `git show HEAD~~~`: display third most recent commit
- `git show main`: display most recent commit on `main`
- `git show HEAD`: display most recent commit
- `git show HEAD~~~`: display third most recent commit
- `git show main`: display most recent commit on `main`
- status
- `git status`: show the working tree status, unstaged and staged files
- `git status`: show the working tree status, unstaged and staged files
[fix-commit]: fixing-commits.md
[git-config-clone]: cloning.html#step-1b-clone-to-your-machine

10
docs/git/collaborate.md
View File

@@ -20,6 +20,7 @@ $ git checkout -b <username>/<branchname>
```
You can choose to rename the branch if you prefer:
```bash
git checkout -b <custombranchname> <username>/<branchname>
```
@@ -31,8 +32,8 @@ pull request locally. GitHub provides a special syntax
([details][github-help-co-pr-locally]) for this since pull requests are
specific to GitHub rather than Git.
First, fetch and create a branch for the pull request, replacing *ID* and
*BRANCHNAME* with the ID of the pull request and your desired branch name:
First, fetch and create a branch for the pull request, replacing _ID_ and
_BRANCHNAME_ with the ID of the pull request and your desired branch name:
```console
$ git fetch upstream pull/ID/head:BRANCHNAME
@@ -47,11 +48,12 @@ $ git checkout BRANCHNAME
Now you work on this branch as you would any other.
Note: you can use the scripts provided in the tools/ directory to fetch pull
requests. You can read more about what they do [here][tools-PR].
requests. You can read more about what they do [here][tools-pr].
```bash
tools/fetch-rebase-pull-request <PR-number>
tools/fetch-pull-request <PR-number>
```
[github-help-co-pr-locally]: https://help.github.com/en/articles/checking-out-pull-requests-locally
[tools-PR]: ../git/zulip-tools.html#fetch-a-pull-request-and-rebase
[tools-pr]: ../git/zulip-tools.html#fetch-a-pull-request-and-rebase

10
docs/git/fixing-commits.md
View File

@@ -1,35 +1,45 @@
# Fixing commits
This is mostly from
[here](https://help.github.com/en/articles/changing-a-commit-message#rewriting-the-most-recent-commit-message).
## Fixing the last commit
### Changing the last commit message
1. `git commit --amend -m "New message"`
### Changing the last commit
1. Make your changes to the files
2. Run `git add <filename>` to add one file or `git add <filename1> <filename2> ...` to add multiple files
3. `git commit --amend`
## Fixing older commits
### Changing commit messages
1. `git rebase -i HEAD~5` (if, for example, you are editing some of the last five commits)
2. For each commit that you want to change the message, change `pick` to `reword`, and save
3. Change the commit messages
### Deleting old commits
1. `git rebase -i HEAD~n` where `n` is the number of commits you are looking at
2. For each commit that you want to delete, change `pick` to `drop`, and save
## Squashing commits
Sometimes, you want to make one commit out of a bunch of commits. To do this,
1. `git rebase -i HEAD~n` where `n` is the number of commits you are interested in
2. Change `pick` to `squash` on the lines containing the commits you want to squash and save
## Reordering commits
1. `git rebase -i HEAD~n` where `n` is the number of commits you are interested in
2. Reorder the lines containing the commits and save
## Pushing commits after tidying them
1. `git push origin +my-feature-branch` (Note the `+` there and substitute your actual branch name.)

12
docs/git/overview.md
View File

@@ -10,7 +10,7 @@ with these details in mind:
[repository][github-zulip], if you are working on something else besides
Zulip server) to your own account and then create feature/issue branches.
When you're ready to get feedback, submit a work-in-progress (WIP) pull
request. *We encourage you to submit WIP pull requests early and often.*
request. _We encourage you to submit WIP pull requests early and often._
- We use a **[rebase][gitbook-rebase]-oriented workflow.** We do not use merge
commits. This means you should use `git fetch` followed by `git rebase`
@@ -24,13 +24,13 @@ with these details in mind:
when another branch is merged, that clutter the commit history (it's
popular with other large projects such as Django). This makes
Zulip's commit history more readable, but a side effect is that many
pull requests we merge will be reported by GitHub's UI as *closed*
instead of *merged*, since GitHub has poor support for
pull requests we merge will be reported by GitHub's UI as _closed_
instead of _merged_, since GitHub has poor support for
rebase-oriented workflows.
- We have a **[code style guide][zulip-rtd-code-style]**, a **[commit message
guide][zulip-rtd-commit-messages]**, and strive for each commit to be *a
minimal coherent idea* (see **[commit
guide][zulip-rtd-commit-messages]**, and strive for each commit to be _a
minimal coherent idea_ (see **[commit
discipline][zulip-rtd-commit-discipline]** for details).
- We provide **many tools to help you submit quality code.** These include
@@ -48,7 +48,7 @@ with these details in mind:
Finally, install the [Zulip developer environment][zulip-rtd-dev-overview], and then
[configure continuous integration for your fork][zulip-git-guide-fork-ci].
***
---
The following sections will help you be awesome with Zulip and Git/GitHub in a
rebased-based workflow. Read through it if you're new to Git, to a rebase-based

6
docs/git/pull-requests.md
View File

@@ -3,7 +3,7 @@
When you're ready for feedback, submit a pull request. Pull requests
are a feature specific to GitHub. They provide a simple, web-based way
to submit your work (often called "patches") to a project. It's called
a *pull request* because you're asking the project to *pull changes*
a _pull request_ because you're asking the project to _pull changes_
from your fork.
If you're unfamiliar with how to create a pull request, you can check
@@ -44,7 +44,7 @@ The best way to update your branch is with `git fetch` and `git rebase`. Do not
use `git pull` or `git merge` as this will create merge commits. See [keep your
fork up to date][keep-up-to-date] for details.
Here's an example (you would replace *issue-123* with the name of your feature branch):
Here's an example (you would replace _issue-123_ with the name of your feature branch):
```console
$ git checkout issue-123
@@ -117,7 +117,7 @@ pull request** button.
Alternatively, if you've recently pushed to your fork, you will see a green
**Compare & pull request** button.
You'll see the *Open a pull request* page:
You'll see the _Open a pull request_ page:
![images-create-pr]

15
docs/git/terminology.md
View File

@@ -2,9 +2,10 @@
When you install Git, it adds a manual entry for `gitglossary`. You can view
this glossary by running `man gitglossary`. Below we've included the Git terms
you'll encounter most often along with their definitions from *gitglossary*.
you'll encounter most often along with their definitions from _gitglossary_.
## branch
A "branch" is an active line of development. The most recent commit
on a branch is referred to as the tip of that branch. The tip of
the branch is referenced by a branch head, which moves forward as
@@ -14,14 +15,17 @@ working tree is associated with just one of them (the "current" or
"checked out" branch), and HEAD points to that branch.
## cache
Obsolete for: index
## checkout
The action of updating all or part of the working tree with a tree
object or blob from the object database, and updating the index and
HEAD if the whole working tree has been pointed at a new branch.
## commit
As a noun: A single point in the Git history; the entire history of
a project is represented as a set of interrelated commits. The word
"commit" is often used by Git in the same places other revision
@@ -33,6 +37,7 @@ state in the Git history, by creating a new commit representing the
current state of the index and advancing HEAD to point at the new
## fast-forward
A fast-forward is a special type of merge where you have a revision
and you are "merging" another branch's changes that happen to be a
descendant of what you have. In such these cases, you do not make a
@@ -41,19 +46,23 @@ happen frequently on a remote-tracking branch of a remote
repository.
## fetch
Fetching a branch means to get the branch's head ref from a remote
repository, to find out which objects are missing from the local
object database, and to get them, too. See also [git-fetch(1)](https://git-scm.com/docs/git-fetch)
## hash
In Git's context, synonym for object name.
## head
A named reference to the commit at the tip of a branch. Heads are
stored in a file in $GIT_DIR/refs/heads/ directory, except when
using packed refs. See also [git-pack-refs(1)](https://git-scm.com/docs/git-pack-refs).
## HEAD
The current branch. In more detail: Your working tree is normally
derived from the state of the tree referred to by HEAD. HEAD is a
reference to one of the heads in your repository, except when using
@@ -61,15 +70,18 @@ a detached HEAD, in which case it directly references an arbitrary
commit.
## index
A collection of files with stat information, whose contents are
stored as objects. The index is a stored version of your working
tree. Truth be told, it can also contain a second, and even a third
version of a working tree, which are used when merging.
## pull
Pulling a branch means to fetch it and merge it. See also [git-pull(1)](https://git-scm.com/docs/git-pull)
## push
Pushing a branch means to get the branch's head ref from a remote
repository, find out if it is a direct ancestor to the branch's
local head ref, and in that case, putting all objects, which are
@@ -79,5 +91,6 @@ the remote head ref. If the remote head is not an ancestor to the
local head, the push fails.
## rebase
To reapply a series of changes from a branch to a different base,
and reset the head of that branch to the result.

18
docs/git/the-git-difference.md
View File

@@ -2,22 +2,22 @@
Whether you're new to Git or have experience with another version control
system (VCS), it's a good idea to learn a bit about how Git works. We recommend
this excellent presentation *[Understanding Git][understanding-git]* from
this excellent presentation _[Understanding Git][understanding-git]_ from
Nelson Elhage and Anders Kaseorg and the [Git Basics][gitbook-basics] chapter
from *Pro Git* by Scott Chacon and Ben Straub.
from _Pro Git_ by Scott Chacon and Ben Straub.
Here are the top things to know:
- **Git works on snapshots.** Unlike other version control systems (e.g.,
Subversion, Perforce, Bazaar), which track files and changes to those files
made over time, Git tracks *snapshots* of your project. Each time you commit
made over time, Git tracks _snapshots_ of your project. Each time you commit
or otherwise make a change to your repository, Git takes a snapshot of your
project and stores a reference to that snapshot. If a file hasn't changed,
Git creates a link to the identical file rather than storing it again.
- **Most Git operations are local.** Git is a distributed version control
system, so once you've cloned a repository, you have a complete copy of that
repository's *entire history*. Staging, committing, branching, and browsing
repository's _entire history_. Staging, committing, branching, and browsing
history are all things you can do locally without network access and without
immediately affecting any remote repositories. To make or receive changes
from remote repositories, you need to `git fetch`, `git pull`, or `git push`.
@@ -45,9 +45,9 @@ Here are the top things to know:
- **Cloning a repository creates a working copy.** Every working copy has a
`.git` subdirectory, which contains its own Git repository. The `.git`
subdirectory also tracks the *index*, a staging area for changes that will
become part of the next commit. All files outside of `.git` is the *working
tree*.
subdirectory also tracks the _index_, a staging area for changes that will
become part of the next commit. All files outside of `.git` is the _working
tree_.
- **Files tracked with Git have possible three states: committed, modified, and
staged.** Committed files are those safely stored in your local `.git`
@@ -56,8 +56,8 @@ Here are the top things to know:
changes but have not yet been marked for inclusion in the next commit; they
have not been added to the index.
- **Git commit workflow is as follows.** Edit files in your *working tree*. Add
to the *index* (that is *stage*) with `git add`. *Commit* to the HEAD of the
- **Git commit workflow is as follows.** Edit files in your _working tree_. Add
to the _index_ (that is _stage_) with `git add`. _Commit_ to the HEAD of the
current branch with `git commit`.
[gitbook-basics]: https://git-scm.com/book/en/v2/Getting-Started-Git-Basics

6
docs/git/troubleshooting.md
View File

@@ -204,10 +204,12 @@ and `>>>>>>>`) markers to indicate where in files there are conflicts.
Tip: You can see recent changes made to a file by running the following
commands:
```bash
git fetch upstream
git log -p upstream/main -- /path/to/file
```
You can use this to compare the changes that you have made to a file with the
ones in upstream, helping you avoid undoing changes from a previous commit when
you are rebasing.
@@ -234,8 +236,8 @@ pay attention and do a bit of work to ensure all of your work is readily
available.
Recall that most Git operations are local. When you commit your changes with
`git commit` they are safely stored in your *local* Git database only. That is,
until you *push* the commits to GitHub, they are only available on the computer
`git commit` they are safely stored in your _local_ Git database only. That is,
until you _push_ the commits to GitHub, they are only available on the computer
where you committed them.
So, before you stop working for the day, or before you switch computers, push

5
docs/git/using.md
View File

@@ -135,7 +135,7 @@ Recall that files tracked with Git have possible three states:
committed, modified, and staged.
To prepare a commit, first add the files with changes that you want
to include in your commit to your staging area. You *add* both new files and
to include in your commit to your staging area. You _add_ both new files and
existing ones. You can also remove files from staging when necessary.
### Get status of working directory
@@ -168,7 +168,7 @@ nothing added to commit but untracked files present (use "git add" to track)
To add changes to your staging area, use `git add <filename>`. Because
`git add` is all about staging the changes you want to commit, you use
it to add *new files* as well as *files with changes* to your staging
it to add _new files_ as well as _files with changes_ to your staging
area.
Continuing our example from above, after we run `git add newfile.py`, we'll see
@@ -188,7 +188,6 @@ view changes to files you haven't yet staged, just use `git diff`.
If you want to add all changes in the working directory, use `git add -A`
([documentation][gitbook-add]).
You can also stage changes using your graphical Git client.
If you stage a file, you can undo it with `git reset HEAD <filename>`. Here's

5
docs/git/zulip-tools.md
View File

@@ -41,7 +41,7 @@ described above in that it does not create a branch for the pull request
checkout.
**This tool checks for uncommitted changes, but it will move the
current branch using `git reset --hard`. Use with caution.**
current branch using `git reset --hard`. Use with caution.**
First, make sure you are working in a branch you want to move (in this
example, we'll use the local `main` branch). Then run the script
@@ -163,10 +163,11 @@ Deleting local branch review-original-5156 (was 5a1e982)
## Merge conflict on yarn.lock file
If there is a merge conflict on yarn.lock, yarn should be run to
regenerate the file. *Important* don't delete the yarn.lock file. Check out the
regenerate the file. _Important_ don't delete the yarn.lock file. Check out the
latest one from `origin/main` so that yarn knows the previous asset versions.
Run the following commands
```bash
git checkout origin/main -- yarn.lock
yarn install

98
docs/overview/architecture-overview.md
View File

@@ -1,8 +1,6 @@
Zulip architectural overview
============================
# Zulip architectural overview
Key codebases
-------------
## Key codebases
The main Zulip codebase is at <https://github.com/zulip/zulip>. It
contains the Zulip backend (written in Python 3.x and Django), the
@@ -37,8 +35,7 @@ translations.
In this overview, we'll mainly discuss the core Zulip server and web
application.
Usage assumptions and concepts
------------------------------
## Usage assumptions and concepts
Zulip is a real-time team chat application meant to provide a great
experience for a wide range of organizations, from companies to
@@ -50,7 +47,7 @@ all modern web browsers, several cross-protocol chat clients, and
numerous dedicated [Zulip API](https://zulip.com/api) clients
(e.g. bots).
A server can host multiple Zulip *realms* (organizations), each on its
A server can host multiple Zulip _realms_ (organizations), each on its
own (sub)domain. While most installations host only one organization, some
such as zulip.com host thousands. Each organization is a private
chamber with its own users, streams, customizations, and so on. This
@@ -61,10 +58,9 @@ more on security considerations and options, see [the security model
section](../production/security-model.md) and the [Zulip Help
Center](https://zulip.com/help).
Components
----------
## Components
![architecture-simple](../images/architecture_simple.png)
![architecture-simple](../images/architecture_simple.png)
### Django and Tornado
@@ -116,8 +112,8 @@ For more details on the frontend, see our documentation on
[directory structure](../overview/directory-structure.md), and
[the static asset pipeline](../subsystems/html-css.html#static-asset-pipeline).
[Jinja2]: http://jinja.pocoo.org/
[Handlebars]: https://handlebarsjs.com/
[jinja2]: http://jinja.pocoo.org/
[handlebars]: https://handlebarsjs.com/
### nginx
@@ -130,16 +126,16 @@ according to the rules laid down in the many config files found in
important of these files. It explains what happens when requests come in
from outside.
- In production, all requests to URLs beginning with `/static/` are
served from the corresponding files in `/home/zulip/prod-static/`,
and the production build process (`tools/build-release-tarball`)
compiles, minifies, and installs the static assets into the
`prod-static/` tree form. In development, files are served directly
from `/static/` in the Git repository.
- Requests to `/json/events` and `/api/v1/events`, i.e. the
real-time push system, are sent to the Tornado server.
- Requests to all other paths are sent to the Django app running via
`uWSGI` via `unix:/home/zulip/deployments/uwsgi-socket`.
- In production, all requests to URLs beginning with `/static/` are
served from the corresponding files in `/home/zulip/prod-static/`,
and the production build process (`tools/build-release-tarball`)
compiles, minifies, and installs the static assets into the
`prod-static/` tree form. In development, files are served directly
from `/static/` in the Git repository.
- Requests to `/json/events` and `/api/v1/events`, i.e. the
real-time push system, are sent to the Tornado server.
- Requests to all other paths are sent to the Django app running via
`uWSGI` via `unix:/home/zulip/deployments/uwsgi-socket`.
- By default (i.e. if `LOCAL_UPLOADS_DIR` is set), nginx will serve
user-uploaded content like avatars, custom emoji, and uploaded
files. However, one can configure Zulip to store these in a cloud
@@ -272,48 +268,48 @@ minimize the set of terminology listed here by giving elements
self-explanatory names.
- **bankruptcy**: When a user has been off Zulip for several days and
has hundreds of unread messages, they are prompted for whether
they want to mark all their unread messages as read. This is
called "declaring bankruptcy" (in reference to the concept in
finance).
has hundreds of unread messages, they are prompted for whether
they want to mark all their unread messages as read. This is
called "declaring bankruptcy" (in reference to the concept in
finance).
- **chevron**: A small downward-facing arrow next to a message's
timestamp, offering contextual options, e.g., "Reply", "Mute [this
topic]", or "Link to this conversation". To avoid visual clutter,
the chevron only appears in the web UI upon hover.
timestamp, offering contextual options, e.g., "Reply", "Mute [this
topic]", or "Link to this conversation". To avoid visual clutter,
the chevron only appears in the web UI upon hover.
- **ellipsis**: A small vertical three dot icon (technically called
as ellipsis-v), present in sidebars as a menu icon.
It offers contextual options for global filters (All messages
and Starred messages), stream filters and topics in left
sidebar and users in right sidebar. To avoid visual clutter
ellipsis only appears in the web UI upon hover.
as ellipsis-v), present in sidebars as a menu icon.
It offers contextual options for global filters (All messages
and Starred messages), stream filters and topics in left
sidebar and users in right sidebar. To avoid visual clutter
ellipsis only appears in the web UI upon hover.
- **huddle**: What the codebase calls a "group private message".
- **message editing**: If the realm admin allows it, then after a user
posts a message, the user has a few minutes to click "Edit" and
change the content of their message. If they do, Zulip adds a
marker such as "(EDITED)" at the top of the message, visible to
anyone who can see the message.
posts a message, the user has a few minutes to click "Edit" and
change the content of their message. If they do, Zulip adds a
marker such as "(EDITED)" at the top of the message, visible to
anyone who can see the message.
- **realm**: What the codebase calls an "organization" in the UI.
- **recipient bar**: A visual indication of the context of a message
or group of messages, displaying the stream and topic or private
message recipient list, at the top of a group of messages. A
typical 1-line message to a new recipient shows to the user as
three lines of content: first the recipient bar, second the
sender's name and avatar alongside the timestamp (and, on hover,
the star and the chevron), and third the message content. The
recipient bar is or contains hyperlinks to help the user narrow.
or group of messages, displaying the stream and topic or private
message recipient list, at the top of a group of messages. A
typical 1-line message to a new recipient shows to the user as
three lines of content: first the recipient bar, second the
sender's name and avatar alongside the timestamp (and, on hover,
the star and the chevron), and third the message content. The
recipient bar is or contains hyperlinks to help the user narrow.
- **star**: Zulip allows a user to mark any message they can see,
public or private, as "starred". A user can easily access messages
they've starred through the "Starred messages" link in the
left sidebar, or use "is:starred" as a narrow or a search
constraint. Whether a user has or has not starred a particular
message is private; other users and realm admins don't know
whether a message has been starred, or by whom.
public or private, as "starred". A user can easily access messages
they've starred through the "Starred messages" link in the
left sidebar, or use "is:starred" as a narrow or a search
constraint. Whether a user has or has not starred a particular
message is private; other users and realm admins don't know
whether a message has been starred, or by whom.
- **subject**: What the codebase calls a "topic" in many places.

88
docs/overview/changelog.md
View File

@@ -749,26 +749,26 @@ details.
#### Upgrade notes for 2.1.0
- The defaults for Zulip's now beta inline URL preview setting have changed.
Previously, the server-level `INLINE_URL_EMBED_PREVIEW` setting was
disabled, and organization-level setting was enabled. Now, the
server-level setting is enabled by default, and the organization-level
setting is disabled. As a result, organization administrators can
configure this feature entirely in the UI. However, servers that had
previously [enabled previews of linked
websites](https://zulip.com/help/allow-image-link-previews) will
lose the setting and need to re-enable it.
Previously, the server-level `INLINE_URL_EMBED_PREVIEW` setting was
disabled, and organization-level setting was enabled. Now, the
server-level setting is enabled by default, and the organization-level
setting is disabled. As a result, organization administrators can
configure this feature entirely in the UI. However, servers that had
previously [enabled previews of linked
websites](https://zulip.com/help/allow-image-link-previews) will
lose the setting and need to re-enable it.
- We rewrote the Google authentication backend to use the
`python-social-auth` system we use for other third-party
authentication systems. For this release, the old variable names
still work, but users should update the following setting names in
their configuration as we will desupport the old names in a future
release:
- In `/etc/zulip/zulip-secrets.conf`, `google_oauth2_client_secret`
is now called with `social_auth_google_secret`.
- In `/etc/zulip/settings.py`, `GOOGLE_OAUTH2_CLIENT_ID` should be
replaced with `SOCIAL_AUTH_GOOGLE_KEY`.
- In `/etc/zulip/settings.py`, `GoogleMobileOauth2Backend` should
be replaced with called `GoogleAuthBackend`.
- In `/etc/zulip/zulip-secrets.conf`, `google_oauth2_client_secret`
is now called with `social_auth_google_secret`.
- In `/etc/zulip/settings.py`, `GOOGLE_OAUTH2_CLIENT_ID` should be
replaced with `SOCIAL_AUTH_GOOGLE_KEY`.
- In `/etc/zulip/settings.py`, `GoogleMobileOauth2Backend` should
be replaced with called `GoogleAuthBackend`.
- Installations using Zulip's LDAP integration without
`LDAP_APPEND_DOMAIN` will need to configure two new settings telling
Zulip how to look up a user in LDAP given their email address:
@@ -807,6 +807,7 @@ lose the setting and need to re-enable it.
downtime, and then upgrade to the new release.
#### Full feature changelog
- Added sortable columns to all tables in settings pages.
- Added webapp support for self-service public data exports.
- Added 'e' keyboard shortcut for editing currently selected message.
@@ -1014,6 +1015,7 @@ lose the setting and need to re-enable it.
### 2.0.0 -- 2019-03-01
#### Highlights
- Added automation for synchronizing user avatars, custom profile
fields, disabled status, and more from LDAP/active directory.
- Added support for explicitly setting oneself as "away" and "user
@@ -1035,14 +1037,15 @@ lose the setting and need to re-enable it.
#### Upgrade notes for 2.0.0
- This release adds support for submitting basic usage statistics to
help the Zulip core team. This feature can be enabled only if a server
is using the [Mobile Push Notification Service][mpns-statistics-docs],
and is enabled by default in that case. To disable it, set
`SUBMIT_USAGE_STATISTICS = False` in `/etc/zulip/settings.py`.
help the Zulip core team. This feature can be enabled only if a server
is using the [Mobile Push Notification Service][mpns-statistics-docs],
and is enabled by default in that case. To disable it, set
`SUBMIT_USAGE_STATISTICS = False` in `/etc/zulip/settings.py`.
[mpns-statistics-docs]: ../production/mobile-push-notifications.html#submitting-statistics
#### Full feature changelog
- Added support for CentOS 7 in the development environment
provisioning process. This is an important step towards production
CentOS/RHEL 7 support.
@@ -1190,6 +1193,7 @@ Zulip installations; it has minimal changes for existing servers.
doing an apt upgrade first.
#### Full feature changelog
- Added an organization setting for message deletion time limits.
- Added an organization setting to control who can edit topics.
- Added Ctrl+K keyboard shortcut for getting to search (same as /, but
@@ -1285,6 +1289,7 @@ Zulip installations; it has minimal changes for existing servers.
### 1.8.0 -- 2018-04-17
#### Highlights
- Dramatically simplified the server installation process; it's now possible
to install Zulip without first setting up outgoing email.
- Added experimental support for importing an organization's history
@@ -1294,8 +1299,8 @@ Zulip installations; it has minimal changes for existing servers.
- Lots of visual polish improvements.
- Countless small bugfixes both in the backend and the UI.
**Security and privacy:**
- Several important security fixes since 1.7.0, which were released
already in 1.7.1 and 1.7.2.
- The security model for private streams has changed. Now
@@ -1312,8 +1317,8 @@ Zulip installations; it has minimal changes for existing servers.
- Added a user setting to control whether email notifications include
message content (or just the fact that there are new messages).
**Visual and UI:**
- Added a user setting to translate emoticons/smileys to emoji.
- Added a user setting to choose the emoji set used in Zulip: Google,
Twitter, Apple, or Emoji One.
@@ -1329,8 +1334,8 @@ Zulip installations; it has minimal changes for existing servers.
- Improved the descriptions and UI for many settings.
- Improved visual design of the help center (/help/).
**Core chat experience:**
- Added support for mentioning groups of users.
- Added a setting to allow users to delete their messages.
- Added support for uploading files in the message-edit UI.
@@ -1390,8 +1395,8 @@ Zulip installations; it has minimal changes for existing servers.
included in email subject lines.
- Fixed uploading user avatars encoded using the CMYK mode.
**User accounts and invites:**
- Added support for users in multiple realms having the same email.
- Added a display for whether the user is logged-in in logged-out
pages.
@@ -1406,8 +1411,8 @@ Zulip installations; it has minimal changes for existing servers.
- Split the Notifications Stream setting in two settings, one for new
users, the other for new streams.
**Stream subscriptions and settings:**
- Added traffic statistics (messages/week) to the "Manage streams" UI.
- Fixed numerous issues in the "stream settings" UI.
- Fixed numerous subtle bugs with the stream creation UI.
@@ -1415,8 +1420,8 @@ Zulip installations; it has minimal changes for existing servers.
so that they can be robust to streams being renamed. The change is
backwards-compatible; existing narrow URLs still work.
**API, bots, and integrations:**
- Rewrote our API documentation to be much more friendly and
expansive; it now covers most important endpoints, with nice examples.
- New integrations: ErrBot, GoCD, Google Code-In, Opbeat, Groove,
@@ -1434,8 +1439,8 @@ Zulip installations; it has minimal changes for existing servers.
numbered-list format.
- APIs for fetching messages now provide more metadata to help clients.
**Keyboard shortcuts:**
- Added new "basics" section to keyboard shortcuts documentation.
- Added a new ">" keyboard shortcut for quote-and-reply.
- Added a new "p" keyboard shortcut to jump to next unread PM thread.
@@ -1443,16 +1448,16 @@ Zulip installations; it has minimal changes for existing servers.
- Changed the hotkey for compose-private-message from "C" to "x".
- Improve keyboard navigation of left and right sidebars with arrow keys.
**Mobile apps backend:**
- Added support for logging into the mobile apps with RemoteUserBackend.
- Improved mobile notifications to support narrowing when one clicks a
mobile push notification.
- Statistics on the fraction of strings that are translated now
include strings in the mobile apps as well.
**For server admins:**
- Added certbot support to the installer for getting certificates.
- Added support for hosting multiple domains, not all as subdomains of
the same base domain.
@@ -1465,8 +1470,8 @@ Zulip installations; it has minimal changes for existing servers.
- Improved Tornado retry logic for connecting to RabbitMQ.
- Added a server setting to control whether digest emails are sent.
**For Zulip developers:**
- Migrated the codebase to use the nice Python 3 typing syntax.
- Added a new /team/ page explaining the team, with a nice
visualization of our contributors.
@@ -1708,28 +1713,28 @@ running a version from before 1.7 should upgrade directly to 1.7.1.
#### Highlights
- A complete visual redesign of the logged-out pages, including login,
registration, integrations, etc.
registration, integrations, etc.
- New visual designs for numerous UI elements, including the emoji
picker, user profile popovers, sidebars, compose, and many more.
picker, user profile popovers, sidebars, compose, and many more.
- A complete redesign of the Zulip settings interfaces to look a lot
nicer and be easier to navigate.
nicer and be easier to navigate.
- Organization admins can now configure the login and registration
pages to show visitors a nice organization profile with custom text
and images, written in Markdown.
pages to show visitors a nice organization profile with custom text
and images, written in Markdown.
- Massively improved performance for presence and settings pages,
especially for very large organizations (1000+ users).
especially for very large organizations (1000+ users).
- A dozen useful new keyboard shortcuts, from editing messages to
emoji reactions to drafts and managing streams.
emoji reactions to drafts and managing streams.
- Typing notifications for private message threads.
- Users can now change their own email address.
- New saved-drafts feature.
- The server can now run on a machine with as little as 2GB of RAM.
- The new [Electron desktop app][electron-app] and new
[React Native mobile app for iOS][ios-app] are now the recommended
Zulip apps.
[React Native mobile app for iOS][ios-app] are now the recommended
Zulip apps.
- Mobile web now works much better, especially on iOS.
- Support for sending mobile push notifications via
[a new forwarding service][mobile-push]
[a new forwarding service][mobile-push]
- Complete translations for Spanish, German, and Czech (and
expanded partial translations for Japanese, Chinese, French,
Hungarian, Polish, Dutch, Russian, Bulgarian, Portuguese,
@@ -2068,6 +2073,7 @@ Zulip apps.
open source.
### 1.3.13 - 2016-06-21
- Added nearly complete internationalization of the Zulip UI.
- Added warning when using @all/@everyone.
- Added button offering to subscribe at bottom of narrows to streams
@@ -2105,6 +2111,7 @@ Zulip apps.
- Removed most of the remaining JavaScript global variables.
### 1.3.12 - 2016-05-10
- CVE-2016-4426: Bot API keys were accessible to other users in the same realm.
- CVE-2016-4427: Deactivated users could access messages if SSO was enabled.
- Fixed a RabbitMQ configuration bug that resulted in reordered messages.
@@ -2112,6 +2119,7 @@ Zulip apps.
- Added an option to logout_all_users to delete only sessions for deactivated users.
### 1.3.11 - 2016-05-02
- Moved email digest support into the default Zulip production configuration.
- Added options for configuring PostgreSQL, RabbitMQ, Redis, and memcached
in settings.py.
@@ -2140,6 +2148,7 @@ Zulip apps.
- Fixed Jira integration incorrectly not @-mentioning assignee.
### 1.3.10 - 2016-01-21
- Added new integration for Travis CI.
- Added settings option to control maximum file upload size.
- Added support for running Zulip development environment in Docker.
@@ -2162,9 +2171,11 @@ Zulip apps.
- Substantially cleaned up console logging from run-dev.py.
### 1.3.9 - 2015-11-16
- Fixed buggy #! lines in upgrade scripts.
### 1.3.8 - 2015-11-15
- Added options to the Python API for working with untrusted server certificates.
- Added a lot of documentation on the development environment and testing.
- Added partial support for translating the Zulip UI.
@@ -2180,6 +2191,7 @@ Zulip apps.
- Major preliminary progress towards supporting Python 3.
### 1.3.7 - 2015-10-19
- Turn off desktop and audible notifications for streams by default.
- Added support for the LDAP authentication integration creating new users.
- Added new endpoint to support Google auth on mobile.

35
docs/overview/directory-structure.md
View File

@@ -30,8 +30,7 @@ paths will be familiar to Django developers.
- `zerver/views/*.py` Most [Django views](https://docs.djangoproject.com/en/1.8/topics/http/views/).
- `zerver/webhooks/` Webhook views and tests for [Zulip's incoming webhook integrations](
https://zulip.com/api/incoming-webhooks-overview).
- `zerver/webhooks/` Webhook views and tests for [Zulip's incoming webhook integrations](https://zulip.com/api/incoming-webhooks-overview).
- `zerver/tornado/views.py` Tornado views.
@@ -41,7 +40,7 @@ paths will be familiar to Django developers.
- `zproject/backends.py` [Authentication backends](https://docs.djangoproject.com/en/1.8/topics/auth/customizing/).
-------------------------------------------------------------------
---
### HTML templates
@@ -53,7 +52,7 @@ templating systems.
- `static/templates/` [Handlebars](https://handlebarsjs.com/) templates for the frontend.
----------------------------------------
---
### JavaScript, TypeScript, and other static assets
@@ -68,9 +67,9 @@ templating systems.
- `node_modules/` Third-party JavaScript installed via `yarn`.
- `static/assets/` For assets not to be served to the web (e.g. the system to
generate our favicons).
generate our favicons).
-----------------------------------------------------------------------
---
### Tests
@@ -82,7 +81,7 @@ templating systems.
- `tools/test-*` Developer-facing test runner scripts.
-----------------------------------------------------
---
### Management commands
@@ -95,10 +94,10 @@ Django context (i.e. with database access).
deactivate a user properly).
- `zilencer/management/commands/` includes some dev-specific
commands such as `populate_db`, which are not included in
the production distribution.
commands such as `populate_db`, which are not included in
the production distribution.
---------------------------------------------------------------
---
### Scripts
@@ -124,7 +123,7 @@ Django context (i.e. with database access).
set up and run our tests in CI. Actual test suites should
go in `tools/`.
---------------------------------------------------------
---
### API and bots
@@ -136,7 +135,7 @@ Django context (i.e. with database access).
- `templates/zerver/integrations/` (within `templates/zerver/`, above).
Documentation for these integrations.
-------------------------------------------------------------------------
---
### Production Puppet configuration
@@ -146,7 +145,7 @@ This is used to deploy essentially all configuration in production.
- `puppet/zulip/manifests/profile/standalone.pp` Main manifest for Zulip standalone deployments.
-----------------------------------------------------------------------
---
### Additional Django apps
@@ -161,25 +160,25 @@ This is used to deploy essentially all configuration in production.
- `zilencer` Primarily used to hold management commands that aren't
used in production. Not included in production distribution.
-----------------------------------------------------------------------
---
### Jinja2 compatibility files
- `zproject/jinja2/__init__.py` Jinja2 environment.
-----------------------------------------------------------------------
---
### Translation files
- `locale/` Backend (Django) and frontend translation data files.
-----------------------------------------------------------------------
---
### Documentation
- `docs/` Source for this documentation.
- `docs/` Source for this documentation.
--------------------------------------------------------------
---
You can consult the repository's `.gitattributes` file to see exactly
which components are excluded from production releases (release

14
docs/overview/release-lifecycle.md
View File

@@ -232,14 +232,8 @@ independently as needed.
[chat-zulip-org]: ../contributing/chat-zulip-org.md
[fork-zulip]: ../production/upgrade-or-modify.html#modifying-zulip
[zulip-server]: https://github.com/zulip/zulip
[mobile-beta]: https://github.com/zulip/zulip-mobile#using-the-beta
[label-blocker]:
https://github.com/zulip/zulip/issues?q=is%3Aissue+is%3Aopen+label%3A%22priority%3A+blocker%22
[label-high]:
https://github.com/zulip/zulip/issues?q=is%3Aissue+is%3Aopen+label%3A%22priority%3A+high%22
[label-release-goal]:
https://github.com/zulip/zulip/issues?q=is%3Aissue+is%3Aopen+label%3A%22release+goal%22
[label-post-release]:
https://github.com/zulip/zulip/issues?q=is%3Aissue+is%3Aopen+label%3A%22post+release%22
[label-blocker]: https://github.com/zulip/zulip/issues?q=is%3Aissue+is%3Aopen+label%3A%22priority%3A+blocker%22
[label-high]: https://github.com/zulip/zulip/issues?q=is%3Aissue+is%3Aopen+label%3A%22priority%3A+high%22
[label-release-goal]: https://github.com/zulip/zulip/issues?q=is%3Aissue+is%3Aopen+label%3A%22release+goal%22
[label-post-release]: https://github.com/zulip/zulip/issues?q=is%3Aissue+is%3Aopen+label%3A%22post+release%22

193
docs/production/authentication-methods.md
View File

@@ -26,6 +26,7 @@ creating the initial realm and user. You can disable it after that.
With just a few lines of configuration, your Zulip server can
authenticate users with any of several single-sign-on (SSO)
authentication providers:
- Google accounts, with `GoogleAuthBackend`
- GitHub accounts, with `GitHubAuthBackend`
- GitLab accounts, with `GitLabAuthBackend`
@@ -38,6 +39,7 @@ are documented in your `settings.py`.
```eval_rst
.. _ldap:
```
## LDAP (including Active Directory)
Zulip supports retrieving information about users via LDAP, and
@@ -51,6 +53,7 @@ In either configuration, you will need to do the following:
organization using LDAP authentication.
1. Tell Zulip how to connect to your LDAP server:
- Fill out the section of your `/etc/zulip/settings.py` headed "LDAP
integration, part 1: Connecting to the LDAP server".
- If a password is required, put it in
@@ -61,6 +64,7 @@ In either configuration, you will need to do the following:
1. Decide how you want to map the information in your LDAP database to
users' account data in Zulip. For each Zulip user, two closely
related concepts are:
- their **email address**. Zulip needs this in order to send, for
example, a notification when they're offline and another user
sends a PM.
@@ -78,32 +82,35 @@ In either configuration, you will need to do the following:
ways to set up the username and/or email mapping:
(A) Using email addresses as Zulip usernames, if LDAP has each
user's email address:
- Make `AUTH_LDAP_USER_SEARCH` a query by email address.
- Set `AUTH_LDAP_REVERSE_EMAIL_SEARCH` to the same query with
`%(email)s` rather than `%(user)s` as the search parameter.
- Set `AUTH_LDAP_USERNAME_ATTR` to the name of the LDAP
attribute for the user's LDAP username in the search result
for `AUTH_LDAP_REVERSE_EMAIL_SEARCH`.
user's email address:
- Make `AUTH_LDAP_USER_SEARCH` a query by email address.
- Set `AUTH_LDAP_REVERSE_EMAIL_SEARCH` to the same query with
`%(email)s` rather than `%(user)s` as the search parameter.
- Set `AUTH_LDAP_USERNAME_ATTR` to the name of the LDAP
attribute for the user's LDAP username in the search result
for `AUTH_LDAP_REVERSE_EMAIL_SEARCH`.
(B) Using LDAP usernames as Zulip usernames, with email addresses
formed consistently like `sam` -> `sam@example.com`:
- Set `AUTH_LDAP_USER_SEARCH` to query by LDAP username
- Set `LDAP_APPEND_DOMAIN = "example.com"`.
formed consistently like `sam` -> `sam@example.com`:
- Set `AUTH_LDAP_USER_SEARCH` to query by LDAP username
- Set `LDAP_APPEND_DOMAIN = "example.com"`.
(C) Using LDAP usernames as Zulip usernames, with email addresses
taken from some other attribute in LDAP (for example, `mail`):
- Set `AUTH_LDAP_USER_SEARCH` to query by LDAP username
- Set `LDAP_EMAIL_ATTR = "mail"`.
- Set `AUTH_LDAP_REVERSE_EMAIL_SEARCH` to a query that will find
an LDAP user given their email address (i.e. a search by
`LDAP_EMAIL_ATTR`). For example:
```python
AUTH_LDAP_REVERSE_EMAIL_SEARCH = LDAPSearch("ou=users,dc=example,dc=com",
ldap.SCOPE_SUBTREE, "(mail=%(email)s)")
```
- Set `AUTH_LDAP_USERNAME_ATTR` to the name of the LDAP
attribute for the user's LDAP username in that search result.
taken from some other attribute in LDAP (for example, `mail`):
- Set `AUTH_LDAP_USER_SEARCH` to query by LDAP username
- Set `LDAP_EMAIL_ATTR = "mail"`.
- Set `AUTH_LDAP_REVERSE_EMAIL_SEARCH` to a query that will find
an LDAP user given their email address (i.e. a search by
`LDAP_EMAIL_ATTR`). For example:
```python
AUTH_LDAP_REVERSE_EMAIL_SEARCH = LDAPSearch("ou=users,dc=example,dc=com",
ldap.SCOPE_SUBTREE, "(mail=%(email)s)")
```
- Set `AUTH_LDAP_USERNAME_ATTR` to the name of the LDAP
attribute for the user's LDAP username in that search result.
You can quickly test whether your configuration works by running:
@@ -119,22 +126,23 @@ email address, if it isn't the same as the "Zulip username").
of the following configurations:
- To access by Active Directory username:
```python
AUTH_LDAP_USER_SEARCH = LDAPSearch("ou=users,dc=example,dc=com",
ldap.SCOPE_SUBTREE, "(sAMAccountName=%(user)s)")
AUTH_LDAP_REVERSE_EMAIL_SEARCH = LDAPSearch("ou=users,dc=example,dc=com",
ldap.SCOPE_SUBTREE, "(mail=%(email)s)")
AUTH_LDAP_USERNAME_ATTR = "sAMAccountName"
```
```python
AUTH_LDAP_USER_SEARCH = LDAPSearch("ou=users,dc=example,dc=com",
ldap.SCOPE_SUBTREE, "(sAMAccountName=%(user)s)")
AUTH_LDAP_REVERSE_EMAIL_SEARCH = LDAPSearch("ou=users,dc=example,dc=com",
ldap.SCOPE_SUBTREE, "(mail=%(email)s)")
AUTH_LDAP_USERNAME_ATTR = "sAMAccountName"
```
- To access by Active Directory email address:
```python
AUTH_LDAP_USER_SEARCH = LDAPSearch("ou=users,dc=example,dc=com",
ldap.SCOPE_SUBTREE, "(mail=%(user)s)")
AUTH_LDAP_REVERSE_EMAIL_SEARCH = LDAPSearch("ou=users,dc=example,dc=com",
ldap.SCOPE_SUBTREE, "(mail=%(email)s)")
AUTH_LDAP_USERNAME_ATTR = "mail"
```
```python
AUTH_LDAP_USER_SEARCH = LDAPSearch("ou=users,dc=example,dc=com",
ldap.SCOPE_SUBTREE, "(mail=%(user)s)")
AUTH_LDAP_REVERSE_EMAIL_SEARCH = LDAPSearch("ou=users,dc=example,dc=com",
ldap.SCOPE_SUBTREE, "(mail=%(email)s)")
AUTH_LDAP_USERNAME_ATTR = "mail"
```
**If you are using LDAP for authentication**: you will need to enable
the `zproject.backends.ZulipLDAPAuthBackend` auth backend, in
@@ -168,6 +176,7 @@ We recommend running this command in a **regular cron job**, to pick
up changes made on your LDAP server.
All of these data synchronization options have the same model:
- New users will be populated automatically with the
name/avatar/etc. from LDAP (as configured) on account creation.
- The `manage.py sync_ldap_user_data` cron job will automatically
@@ -270,6 +279,7 @@ the fields that would be useful to sync from your LDAP databases.
### Multiple LDAP searches
To do the union of multiple LDAP searches, use `LDAPSearchUnion`. For example:
```python
AUTH_LDAP_USER_SEARCH = LDAPSearchUnion(
LDAPSearch("ou=users,dc=example,dc=com", ldap.SCOPE_SUBTREE, "(uid=%(user)s)"),
@@ -300,6 +310,7 @@ For the root subdomain, `www` in the list will work, or any other of
For example, with `org_membership` set to `department`, a user with
the following attributes will have access to the root and `engineering` subdomains:
```text
...
department: engineering
@@ -364,26 +375,27 @@ it as follows:
1. Tell your IdP how to find your Zulip server:
- **SP Entity ID**: `https://yourzulipdomain.example.com`.
- **SP Entity ID**: `https://yourzulipdomain.example.com`.
The `Entity ID` should match the value of
`SOCIAL_AUTH_SAML_SP_ENTITY_ID` computed in the Zulip settings.
You can get the correct value by running the following:
`/home/zulip/deployments/current/scripts/get-django-setting SOCIAL_AUTH_SAML_SP_ENTITY_ID`.
The `Entity ID` should match the value of
`SOCIAL_AUTH_SAML_SP_ENTITY_ID` computed in the Zulip settings.
You can get the correct value by running the following:
`/home/zulip/deployments/current/scripts/get-django-setting SOCIAL_AUTH_SAML_SP_ENTITY_ID`.
- **SSO URL**:
`https://yourzulipdomain.example.com/complete/saml/`. This is
the "SAML ACS url" in SAML terminology.
- **SSO URL**:
`https://yourzulipdomain.example.com/complete/saml/`. This is
the "SAML ACS url" in SAML terminology.
If you're
[hosting multiple organizations](../production/multiple-organizations.html#authentication),
you need to use `SOCIAL_AUTH_SUBDOMAIN`. For example,
if `SOCIAL_AUTH_SUBDOMAIN="auth"` and `EXTERNAL_HOST=zulip.example.com`,
this should be `https://auth.zulip.example.com/complete/saml/`.
If you're
[hosting multiple organizations](../production/multiple-organizations.html#authentication),
you need to use `SOCIAL_AUTH_SUBDOMAIN`. For example,
if `SOCIAL_AUTH_SUBDOMAIN="auth"` and `EXTERNAL_HOST=zulip.example.com`,
this should be `https://auth.zulip.example.com/complete/saml/`.
2. Tell Zulip how to connect to your SAML provider(s) by filling
1. Tell Zulip how to connect to your SAML provider(s) by filling
out the section of `/etc/zulip/settings.py` on your Zulip server
with the heading "SAML Authentication".
- You will need to update `SOCIAL_AUTH_SAML_ORG_INFO` with your
organization name (`displayname` may appear in the IdP's
authentication flow; `name` won't be displayed to humans).
@@ -411,29 +423,29 @@ it as follows:
6. The `auto_signup` field determines how Zulip should handle
login attempts by users who don't have an account yet.
3. Install the certificate(s) required for SAML authentication. You
will definitely need the public certificate of your IdP. Some IdP
providers also support the Zulip server (Service Provider) having
a certificate used for encryption and signing. We detail these
steps as optional below, because they aren't required for basic
setup, and some IdPs like Okta don't fully support Service
Provider certificates. You should install them as follows:
1. Install the certificate(s) required for SAML authentication. You
will definitely need the public certificate of your IdP. Some IdP
providers also support the Zulip server (Service Provider) having
a certificate used for encryption and signing. We detail these
steps as optional below, because they aren't required for basic
setup, and some IdPs like Okta don't fully support Service
Provider certificates. You should install them as follows:
1. On your Zulip server, `mkdir -p /etc/zulip/saml/idps/`
2. Put the IDP public certificate in `/etc/zulip/saml/idps/{idp_name}.crt`
3. (Optional) Put the Zulip server public certificate in `/etc/zulip/saml/zulip-cert.crt`
and the corresponding private key in `/etc/zulip/saml/zulip-private-key.key`. Note that
the certificate should be the single X.509 certificate for the server, not a full chain of
trust, which consists of multiple certificates.
4. Set the proper permissions on these files and directories:
1. On your Zulip server, `mkdir -p /etc/zulip/saml/idps/`
2. Put the IDP public certificate in `/etc/zulip/saml/idps/{idp_name}.crt`
3. (Optional) Put the Zulip server public certificate in `/etc/zulip/saml/zulip-cert.crt`
and the corresponding private key in `/etc/zulip/saml/zulip-private-key.key`. Note that
the certificate should be the single X.509 certificate for the server, not a full chain of
trust, which consists of multiple certificates.
4. Set the proper permissions on these files and directories:
```bash
chown -R zulip.zulip /etc/zulip/saml/
find /etc/zulip/saml/ -type f -exec chmod 644 -- {} +
chmod 640 /etc/zulip/saml/zulip-private-key.key
```
```bash
chown -R zulip.zulip /etc/zulip/saml/
find /etc/zulip/saml/ -type f -exec chmod 644 -- {} +
chmod 640 /etc/zulip/saml/zulip-private-key.key
```
4. (Optional) If you configured the optional public and private server
1. (Optional) If you configured the optional public and private server
certificates above, you can enable the additional setting
`"authnRequestsSigned": True` in `SOCIAL_AUTH_SAML_SECURITY_CONFIG`
to have the SAMLRequests the server will be issuing to the IdP
@@ -442,18 +454,18 @@ it as follows:
assertions in the SAMLResponses the IdP will send about
authenticated users.
5. Enable the `zproject.backends.SAMLAuthBackend` auth backend, in
`AUTHENTICATION_BACKENDS` in `/etc/zulip/settings.py`.
1. Enable the `zproject.backends.SAMLAuthBackend` auth backend, in
`AUTHENTICATION_BACKENDS` in `/etc/zulip/settings.py`.
6. [Restart the Zulip server](../production/settings.md) to ensure
your settings changes take effect. The Zulip login page should now
have a button for SAML authentication that you can use to log in or
create an account (including when creating a new organization).
1. [Restart the Zulip server](../production/settings.md) to ensure
your settings changes take effect. The Zulip login page should now
have a button for SAML authentication that you can use to log in or
create an account (including when creating a new organization).
7. If the configuration was successful, the server's metadata can be
found at `https://yourzulipdomain.example.com/saml/metadata.xml`. You
can use this for verifying your configuration or provide it to your
IdP.
1. If the configuration was successful, the server's metadata can be
found at `https://yourzulipdomain.example.com/saml/metadata.xml`. You
can use this for verifying your configuration or provide it to your
IdP.
[saml-help-center]: https://zulip.com/help/saml-authentication
@@ -502,7 +514,6 @@ to the root and `engineering` subdomains:
</saml2:Attribute>
```
## Apache-based SSO with `REMOTE_USER`
If you have any existing SSO solution where a preferred way to deploy
@@ -542,6 +553,7 @@ straightforward way to deploy that SSO solution with Zulip.
using the `htpasswd` example configuration and demonstrate that
working end-to-end, before returning later to configure your SSO
solution. You can do that with the following steps:
```bash
/home/zulip/deployments/current/scripts/restart-server
cd /etc/apache2/sites-available/
@@ -578,11 +590,11 @@ improve this SSO setup documentation are very welcome!
- The following log files can be helpful when debugging this setup:
- `/var/log/zulip/{errors.log,server.log}` (the usual places)
- `/var/log/nginx/access.log` (nginx access logs)
- `/var/log/apache2/zulip_auth_access.log` (from the
`zulip-sso.conf` Apache config file; you may want to change
`LogLevel` in that file to "debug" to make this more verbose)
- `/var/log/zulip/{errors.log,server.log}` (the usual places)
- `/var/log/nginx/access.log` (nginx access logs)
- `/var/log/apache2/zulip_auth_access.log` (from the
`zulip-sso.conf` Apache config file; you may want to change
`LogLevel` in that file to "debug" to make this more verbose)
### Life of an Apache-based SSO login attempt
@@ -594,7 +606,7 @@ to debug.
- Since you've configured `/etc/zulip/settings.py` to only define the
`zproject.backends.ZulipRemoteUserBackend`,
`zproject/computed_settings.py` configures `/accounts/login/sso/` as
`HOME_NOT_LOGGED_IN`. This makes `https://zulip.example.com/`
`HOME_NOT_LOGGED_IN`. This makes `https://zulip.example.com/`
(a.k.a. the homepage for the main Zulip Django app running behind
nginx) redirect to `/accounts/login/sso/` for a user that isn't
logged in.
@@ -626,9 +638,9 @@ Zulip supports using the web flow for Sign in with Apple on
self-hosted servers. To do so, you'll need to do the following:
1. Visit [the Apple Developer site][apple-developer] and [Create a
Services ID.][apple-create-services-id]. When prompted for a "Return
URL", enter `https://zulip.example.com/complete/apple/` (using the
domain for your server).
Services ID.][apple-create-services-id]. When prompted for a "Return
URL", enter `https://zulip.example.com/complete/apple/` (using the
domain for your server).
1. Create a [Sign in with Apple private key][apple-create-private-key].
@@ -642,6 +654,7 @@ domain for your server).
```
1. Configure Apple authentication in `/etc/zulip/settings.py`:
- `SOCIAL_AUTH_APPLE_TEAM`: Your Team ID from Apple, which is a
string like "A1B2C3D4E5".
- `SOCIAL_AUTH_APPLE_SERVICES_ID`: The Services ID you created in

194
docs/production/deployment.md
View File

@@ -191,15 +191,15 @@ configure that as follows:
with `/home/zulip/deployments/current/scripts/restart-server`.
1. Add the following block to `/etc/zulip/zulip.conf`:
```ini
[application_server]
nginx_listen_port = 12345
```
```ini
[application_server]
nginx_listen_port = 12345
```
1. As root, run
`/home/zulip/deployments/current/scripts/zulip-puppet-apply`. This
will convert Zulip's main `nginx` configuration file to use your new
port.
`/home/zulip/deployments/current/scripts/zulip-puppet-apply`. This
will convert Zulip's main `nginx` configuration file to use your new
port.
We also have documentation for a Zulip server [using HTTP][using-http] for use
behind reverse proxies.
@@ -219,23 +219,24 @@ To use Smokescreen:
1. Add `, zulip::profile::smokescreen` to the list of `puppet_classes`
in `/etc/zulip/zulip.conf`. A typical value after this change is:
```ini
puppet_classes = zulip::profile::standalone, zulip::profile::smokescreen
```
```ini
puppet_classes = zulip::profile::standalone, zulip::profile::smokescreen
```
1. Optionally, configure the [smokescreen ACLs][smokescreen-acls]. By
default, Smokescreen denies access to all [non-public IP
addresses](https://en.wikipedia.org/wiki/Private_network), including
127.0.0.1.
default, Smokescreen denies access to all [non-public IP
addresses](https://en.wikipedia.org/wiki/Private_network), including
127.0.0.1.
1. Add the following block to `/etc/zulip/zulip.conf`, substituting in
your proxy's hostname/IP and port:
```ini
[http_proxy]
host = 127.0.0.1
port = 4750
```
```ini
[http_proxy]
host = 127.0.0.1
port = 4750
```
1. If you intend to also make the Smokescreen install available to
other hosts, set `listen_address` in the same block. Note that you
@@ -280,22 +281,23 @@ HTTP as follows:
1. Add the following block to `/etc/zulip/zulip.conf`:
```ini
[application_server]
http_only = true
```
```ini
[application_server]
http_only = true
```
1. As root, run
`/home/zulip/deployments/current/scripts/zulip-puppet-apply`. This
will convert Zulip's main `nginx` configuration file to allow HTTP
instead of HTTPS.
`/home/zulip/deployments/current/scripts/zulip-puppet-apply`. This
will convert Zulip's main `nginx` configuration file to allow HTTP
instead of HTTPS.
1. Finally, restart the Zulip server, using
`/home/zulip/deployments/current/scripts/restart-server`.
`/home/zulip/deployments/current/scripts/restart-server`.
### nginx configuration
For `nginx` configuration, there's two things you need to set up:
- The root `nginx.conf` file. We recommend using
`/etc/nginx/nginx.conf` from your Zulip server for our recommended
settings. E.g. if you don't set `client_max_body_size`, it won't be
@@ -341,12 +343,12 @@ make the following changes in two configuration files.
1. Follow the instructions for [Configure Zulip to allow HTTP](#configuring-zulip-to-allow-http).
2. Add the following to `/etc/zulip/settings.py`:
```python
EXTERNAL_HOST = 'zulip.example.com'
ALLOWED_HOSTS = ['zulip.example.com', '127.0.0.1']
USE_X_FORWARDED_HOST = True
```
```python
EXTERNAL_HOST = 'zulip.example.com'
ALLOWED_HOSTS = ['zulip.example.com', '127.0.0.1']
USE_X_FORWARDED_HOST = True
```
3. Restart your Zulip server with `/home/zulip/deployments/current/scripts/restart-server`.
@@ -357,41 +359,41 @@ make the following changes in two configuration files.
and then run
`a2ensite zulip.example.com && systemctl reload apache2`):
```apache
<VirtualHost *:80>
ServerName zulip.example.com
RewriteEngine On
RewriteRule ^ https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]
</VirtualHost>
```apache
<VirtualHost *:80>
ServerName zulip.example.com
RewriteEngine On
RewriteRule ^ https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]
</VirtualHost>
<VirtualHost *:443>
ServerName zulip.example.com
<VirtualHost *:443>
ServerName zulip.example.com
RequestHeader set "X-Forwarded-Proto" expr=%{REQUEST_SCHEME}
RequestHeader set "X-Forwarded-SSL" expr=%{HTTPS}
RequestHeader set "X-Forwarded-Proto" expr=%{REQUEST_SCHEME}
RequestHeader set "X-Forwarded-SSL" expr=%{HTTPS}
RewriteEngine On
RewriteRule /(.*) http://localhost:5080/$1 [P,L]
RewriteEngine On
RewriteRule /(.*) http://localhost:5080/$1 [P,L]
<Location />
Require all granted
ProxyPass http://localhost:5080/ timeout=300
ProxyPassReverse http://localhost:5080/
ProxyPassReverseCookieDomain 127.0.0.1 zulip.example.com
</Location>
<Location />
Require all granted
ProxyPass http://localhost:5080/ timeout=300
ProxyPassReverse http://localhost:5080/
ProxyPassReverseCookieDomain 127.0.0.1 zulip.example.com
</Location>
SSLEngine on
SSLProxyEngine on
SSLCertificateFile /etc/letsencrypt/live/zulip.example.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/zulip.example.com/privkey.pem
SSLOpenSSLConfCmd DHParameters "/etc/nginx/dhparam.pem"
SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1
SSLCipherSuite ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
SSLHonorCipherOrder off
SSLSessionTickets off
Header set Strict-Transport-Security "max-age=31536000"
</VirtualHost>
```
SSLEngine on
SSLProxyEngine on
SSLCertificateFile /etc/letsencrypt/live/zulip.example.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/zulip.example.com/privkey.pem
SSLOpenSSLConfCmd DHParameters "/etc/nginx/dhparam.pem"
SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1
SSLCipherSuite ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
SSLHonorCipherOrder off
SSLSessionTickets off
Header set Strict-Transport-Security "max-age=31536000"
</VirtualHost>
```
### HAProxy configuration
@@ -416,16 +418,16 @@ If you're using another reverse proxy implementation, there are few
things you need to be careful about when configuring it:
1. Configure your reverse proxy (or proxies) to correctly maintain the
`X-Forwarded-For` HTTP header, which is supposed to contain the series
of IP addresses the request was forwarded through. You can verify
your work by looking at `/var/log/zulip/server.log` and checking it
has the actual IP addresses of clients, not the IP address of the
proxy server.
`X-Forwarded-For` HTTP header, which is supposed to contain the series
of IP addresses the request was forwarded through. You can verify
your work by looking at `/var/log/zulip/server.log` and checking it
has the actual IP addresses of clients, not the IP address of the
proxy server.
2. Ensure your proxy doesn't interfere with Zulip's use of
long-polling for real-time push from the server to your users'
browsers. This [nginx code snippet][nginx-proxy-longpolling-config]
does this.
long-polling for real-time push from the server to your users'
browsers. This [nginx code snippet][nginx-proxy-longpolling-config]
does this.
The key configuration options are, for the `/json/events` and
`/api/1/events` endpoints:
@@ -436,14 +438,13 @@ The key configuration options are, for the `/json/events` and
return occasional 502 errors to clients using Zulip's events API.
3. The other tricky failure mode we've seen with `nginx` reverse
proxies is that they can load-balance between the IPv4 and IPv6
addresses for a given hostname. This can result in mysterious errors
that can be quite difficult to debug. Be sure to declare your
`upstreams` equivalent in a way that won't do load-balancing
unexpectedly (e.g. pointing to a DNS name that you haven't configured
with multiple IPs for your Zulip machine; sometimes this happens with
IPv6 configuration).
proxies is that they can load-balance between the IPv4 and IPv6
addresses for a given hostname. This can result in mysterious errors
that can be quite difficult to debug. Be sure to declare your
`upstreams` equivalent in a way that won't do load-balancing
unexpectedly (e.g. pointing to a DNS name that you haven't configured
with multiple IPs for your Zulip machine; sometimes this happens with
IPv6 configuration).
## System and deployment configuration
@@ -461,11 +462,12 @@ The most common is **`zulip::profile::standalone`**, used for a
stand-alone single-host deployment.
[Components](../overview/architecture-overview.html#components) of
that include:
- **`zulip::profile::app_frontend`**
- **`zulip::profile::memcached`**
- **`zulip::profile::postgresql`**
- **`zulip::profile::redis`**
- **`zulip::profile::rabbitmq`**
- **`zulip::profile::app_frontend`**
- **`zulip::profile::memcached`**
- **`zulip::profile::postgresql`**
- **`zulip::profile::redis`**
- **`zulip::profile::rabbitmq`**
If you are using a [Apache as a single-sign-on
authenticator](../production/authentication-methods.html#apache-based-sso-with-remote-user),
@@ -477,8 +479,6 @@ Set to the string `enabled` if enabling the [multi-language PGroonga
search
extension](../subsystems/full-text-search.html#multi-language-full-text-search).
### `[deployment]`
#### `deploy_options`
@@ -486,12 +486,12 @@ extension](../subsystems/full-text-search.html#multi-language-full-text-search).
Options passed by `upgrade-zulip` and `upgrade-zulip-from-git` into
`upgrade-zulip-stage-2`. These might be any of:
- **`--skip-puppet`** skips doing Puppet/apt upgrades. The user will need
to run `zulip-puppet-apply` manually after the upgrade.
- **`--skip-migrations`** skips running database migrations. The
user will need to run `./manage.py migrate` manually after the upgrade.
- **`--skip-purge-old-deployments`** skips purging old deployments;
without it, only deployments with the last two weeks are kept.
- **`--skip-puppet`** skips doing Puppet/apt upgrades. The user will need
to run `zulip-puppet-apply` manually after the upgrade.
- **`--skip-migrations`** skips running database migrations. The
user will need to run `./manage.py migrate` manually after the upgrade.
- **`--skip-purge-old-deployments`** skips purging old deployments;
without it, only deployments with the last two weeks are kept.
Generally installations will not want to set any of these options; the
`--skip-*` options are primarily useful for reducing upgrade downtime
@@ -502,8 +502,6 @@ for servers that are upgraded frequently by core Zulip developers.
Default repository URL used when [upgrading from a Git
repository](../production/upgrade-or-modify.html#upgrading-from-a-git-repository).
### `[application_server]`
#### `http_only`
@@ -553,8 +551,6 @@ Override the default uwsgi backlog of 128 connections.
Override the default `uwsgi` (Django) process count of 6 on hosts with
more than 3.5GiB of RAM, 4 on hosts with less.
### `[certbot]`
#### `auto_renew`
@@ -563,8 +559,6 @@ If set to the string `yes`, [Certbot will attempt to automatically
renew its certificate](../production/ssl-certificates.html#certbot-recommended). Do
no set by hand; use `scripts/setup/setup-certbot` to configure this.
### `[postfix]`
#### `mailname`
@@ -615,16 +609,12 @@ connections.
The version of PostgreSQL that is in use. Do not set by hand; use the
[PostgreSQL upgrade tool](../production/upgrade-or-modify.html#upgrading-postgresql).
### `[rabbitmq]`
#### `nodename`
The name used to identify the local RabbitMQ server; do not modify.
### `[memcached]`
#### `memory`
@@ -632,8 +622,6 @@ The name used to identify the local RabbitMQ server; do not modify.
Override the number of megabytes of memory that memcached should be
configured to consume; defaults to 1/8th of the total server memory.
### `[loadbalancer]`
#### `ips`
@@ -641,8 +629,6 @@ configured to consume; defaults to 1/8th of the total server memory.
Comma-separated list of IP addresses or netmasks of external
load balancers whose `X-Forwarded-For` should be respected.
### `[http_proxy]`
#### `host`

58
docs/production/email-gateway.md
View File

@@ -18,10 +18,10 @@ that address will be delivered into the stream.
There are two ways to configure Zulip's email gateway:
1. Local delivery (recommended): A postfix server runs on the Zulip
server and passes the emails directly to Zulip.
1. Polling: A cron job running on the Zulip server checks an IMAP
inbox (`username@example.com`) every minute for new emails.
1. Local delivery (recommended): A postfix server runs on the Zulip
server and passes the emails directly to Zulip.
1. Polling: A cron job running on the Zulip server checks an IMAP
inbox (`username@example.com`) every minute for new emails.
The local delivery configuration is preferred for production because
it supports nicer looking email addresses and has no cron delay. The
@@ -57,31 +57,32 @@ using an [HTTP reverse proxy][reverse-proxy]).
configuring email for `emaildomain.example.com` to be processed by
`hostname.example.com`. You can check your work using this command:
```console
$ dig +short emaildomain.example.com -t MX
1 hostname.example.com
```
```console
$ dig +short emaildomain.example.com -t MX
1 hostname.example.com
```
1. Log in to your Zulip server; the remaining steps all happen there.
1. Add `, zulip::postfix_localmail` to `puppet_classes` in
`/etc/zulip/zulip.conf`. A typical value after this change is:
```ini
puppet_classes = zulip::profile::standalone, zulip::postfix_localmail
```
1. If `hostname.example.com` is different from
1. If `hostname.example.com` is different from
`emaildomain.example.com`, add a section to `/etc/zulip/zulip.conf`
on your Zulip server like this:
```ini
[postfix]
mailname = emaildomain.example.com
```
```ini
[postfix]
mailname = emaildomain.example.com
```
This tells postfix to expect to receive emails at addresses ending
with `@emaildomain.example.com`, overriding the default of
`@hostname.example.com`.
This tells postfix to expect to receive emails at addresses ending
with `@emaildomain.example.com`, overriding the default of
`@hostname.example.com`.
1. Run `/home/zulip/deployments/current/scripts/zulip-puppet-apply`
(and answer `y`) to apply your new `/etc/zulip/zulip.conf`
@@ -100,26 +101,27 @@ Congratulations! The integration should be fully operational.
## Polling setup
1. Create an email account dedicated to Zulip's email gateway
messages. We assume the address is of the form
`username@example.com`. The email provider needs to support the
standard model of delivering emails sent to
`username+stuff@example.com` to the `username@example.com` inbox.
messages. We assume the address is of the form
`username@example.com`. The email provider needs to support the
standard model of delivering emails sent to
`username+stuff@example.com` to the `username@example.com` inbox.
1. Edit `/etc/zulip/settings.py`, and set `EMAIL_GATEWAY_PATTERN` to
`"username+%s@example.com"`.
1. Set up IMAP for your email account and obtain the authentication details.
([Here's how it works with Gmail](https://support.google.com/mail/answer/7126229?hl=en))
([Here's how it works with Gmail](https://support.google.com/mail/answer/7126229?hl=en))
1. Configure IMAP access in the appropriate Zulip settings:
- Login and server connection details in `/etc/zulip/settings.py`
in the email gateway integration section (`EMAIL_GATEWAY_LOGIN` and others).
- Password in `/etc/zulip/zulip-secrets.conf` as `email_gateway_password`.
- Login and server connection details in `/etc/zulip/settings.py`
in the email gateway integration section (`EMAIL_GATEWAY_LOGIN` and others).
- Password in `/etc/zulip/zulip-secrets.conf` as `email_gateway_password`.
1. Install a cron job to poll the inbox every minute for new messages:
```bash
cd /home/zulip/deployments/current/
sudo cp puppet/zulip/files/cron.d/email-mirror /etc/cron.d/
```
```bash
cd /home/zulip/deployments/current/
sudo cp puppet/zulip/files/cron.d/email-mirror /etc/cron.d/
```
Congratulations! The integration should be fully operational.

1
docs/production/email.md
View File

@@ -100,6 +100,7 @@ We don't recommend using an inbox product like Gmail for outgoing
email, because Gmail's anti-spam measures make this annoying. But if
you want to use a Gmail account to send outgoing email anyway, here's
how to make it work:
- Create a totally new Gmail account for your Zulip server; you don't
want Zulip's automated emails to come from your personal email address.
- If you're using 2-factor authentication on the Gmail account, you'll

6
docs/production/expensive-migrations.md
View File

@@ -13,10 +13,10 @@ you'd like to watch the downtime phase of the upgrade closely, you
can run them manually before starting the upgrade:
1. Log in to your Zulip server as the `zulip` user (or as `root` and
then run `su zulip` to drop privileges), and
`cd /home/zulip/deployments/current`
then run `su zulip` to drop privileges), and
`cd /home/zulip/deployments/current`
2. Run `./manage.py dbshell`. This will open a shell connected to the
PostgreSQL database.
PostgreSQL database.
3. In the PostgreSQL shell, run the following commands:
```postgresql

70
docs/production/export-and-import.md
View File

@@ -65,6 +65,7 @@ su zulip -c '/home/zulip/deployments/current/manage.py backup'
```
The backup tool provides the following options:
- `--output=/tmp/backup.tar.gz`: Filename to write the backup tarball
to (default: write to a file in `/tmp`). On success, the
console output will show the path to the output tarball.
@@ -160,12 +161,12 @@ included in the backups generated by Zulip's standard tools. The
data includes:
- The PostgreSQL database. You can back this up with any standard
database export or backup tool. Zulip has built-in support for taking
daily incremental backups using
[wal-g](https://github.com/wal-g/wal-g); these backups are stored for
30 days in S3. If you have an Amazon S3 bucket you wish to store for
storing the backups, edit `/etc/zulip/zulip-secrets.conf` on the
PostgreSQL server to add:
database export or backup tool. Zulip has built-in support for taking
daily incremental backups using
[wal-g](https://github.com/wal-g/wal-g); these backups are stored for
30 days in S3. If you have an Amazon S3 bucket you wish to store for
storing the backups, edit `/etc/zulip/zulip-secrets.conf` on the
PostgreSQL server to add:
```ini
s3_backups_key = # aws public key
@@ -180,17 +181,17 @@ PostgreSQL server to add:
`/usr/lib/nagios/plugins/zulip_postgresql_backups/check_postgresql_backup`.
- Any user-uploaded files. If you're using S3 as storage for file
uploads, this is backed up in S3. But if you have instead set
`LOCAL_UPLOADS_DIR`, any files uploaded by users (including avatars)
will be stored in that directory and you'll want to back it up.
uploads, this is backed up in S3. But if you have instead set
`LOCAL_UPLOADS_DIR`, any files uploaded by users (including avatars)
will be stored in that directory and you'll want to back it up.
- Your Zulip configuration including secrets from `/etc/zulip/`.
E.g. if you lose the value of `secret_key`, all users will need to
log in again when you set up a replacement server since you won't be
able to verify their cookies. If you lose `avatar_salt`, any
user-uploaded avatars will need to be re-uploaded (since avatar
filenames are computed using a hash of `avatar_salt` and user's
email), etc.
E.g. if you lose the value of `secret_key`, all users will need to
log in again when you set up a replacement server since you won't be
able to verify their cookies. If you lose `avatar_salt`, any
user-uploaded avatars will need to be re-uploaded (since avatar
filenames are computed using a hash of `avatar_salt` and user's
email), etc.
[export-import]: ../production/export-and-import.md
@@ -258,17 +259,17 @@ before exporting; so that nobody can send new messages (etc.) while
you're exporting data. There are two ways to do this:
1. `./scripts/stop-server`, which stops the whole server. This is
preferred if you're not hosting multiple organizations, because it has
no side effects other than disabling the Zulip server for the
duration.
preferred if you're not hosting multiple organizations, because it has
no side effects other than disabling the Zulip server for the
duration.
1. Pass `--deactivate` to `./manage export`, which first deactivates
the target organization, logging out all active login sessions and
preventing all accounts from logging in or accessing the API. This is
preferred for environments like Zulip Cloud where you might want to
export a single organization without disrupting any other users, and
the intent is to move hosting of the organization (and forcing users
to re-log in would be required as part of the hosting migration
anyway).
the target organization, logging out all active login sessions and
preventing all accounts from logging in or accessing the API. This is
preferred for environments like Zulip Cloud where you might want to
export a single organization without disrupting any other users, and
the intent is to move hosting of the organization (and forcing users
to re-log in would be required as part of the hosting migration
anyway).
We include both options in the instructions below, commented out so
that neither runs (using the `# ` at the start of the lines). If
@@ -298,8 +299,9 @@ archive of all the organization's uploaded files.
## Import into a new Zulip server
1. [Install a new Zulip server](../production/install.md),
**skipping Step 3** (you'll create your Zulip organization via the data
import tool instead).
**skipping Step 3** (you'll create your Zulip organization via the data
import tool instead).
- Ensure that the Zulip server you're importing into is running the same
version of Zulip as the server you're exporting from.
@@ -321,10 +323,10 @@ archive of all the organization's uploaded files.
budget extra RAM for running the data import tool.
2. If your new Zulip server is meant to fully replace a previous Zulip
server, you may want to copy some settings from `/etc/zulip` to your
new server to reuse the server-level configuration and secret keys
from your old server. There are a few important details to understand
about doing so:
server, you may want to copy some settings from `/etc/zulip` to your
new server to reuse the server-level configuration and secret keys
from your old server. There are a few important details to understand
about doing so:
- Copying `/etc/zulip/settings.py` and `/etc/zulip/zulip.conf` is
safe and recommended. Care is required when copying secrets from
@@ -361,8 +363,8 @@ about doing so:
avatar image improperly for users whose ID was renumbered.
3. Log in to a shell on your Zulip server as the `zulip` user. Run the
following commands, replacing the filename with the path to your data
export tarball:
following commands, replacing the filename with the path to your data
export tarball:
```bash
cd ~
@@ -408,6 +410,7 @@ recommend starting with sending one to yourself for testing:
```
and then once you're ready, you can email them to everyone using e.g.
```bash
./manage.py send_password_reset_email -r '' --all-users
```
@@ -434,6 +437,7 @@ realm.delete()
The output contains details on the objects deleted from the database.
Now, exit the management shell and run this to clear Zulip's cache:
```bash
/home/zulip/deployments/current/scripts/setup/flush-memcached
```

2
docs/production/giphy-gif-integration.md
View File

@@ -41,7 +41,5 @@ Zulip server. Your users can now use the integration as described in
[the help center article][help-center-giphy]. (A browser reload may
be required).
[help-center-giphy]: https://zulip.com/help/animated-gifs-from-giphy
[giphy-dashboard]: https://developers.giphy.com/dashboard/

2
docs/production/install-existing-server.md
View File

@@ -11,7 +11,7 @@ configuration files in /etc, so we recommend against installing it on
a server running other nginx or django apps.
But if you do, here are some things you can do that may make it
possible to retain your existing site. However, this is *NOT*
possible to retain your existing site. However, this is _NOT_
recommended, and you may break your server. Make sure you have backups
and a provisioning script ready to go to wipe and restore your
existing services if (when) your server goes down.

37
docs/production/install.md
View File

@@ -25,10 +25,10 @@ tar -xf zulip-server-latest.tar.gz
```
- If you'd like to verify the download, we
[publish the sha256sums of our release tarballs](https://www.zulip.org/dist/releases/SHA256SUMS.txt).
[publish the sha256sums of our release tarballs](https://www.zulip.org/dist/releases/SHA256SUMS.txt).
- You can also
[install a pre-release version of Zulip](../production/deployment.html#installing-zulip-from-git)
using code from our [repository on GitHub](https://github.com/zulip/zulip/).
[install a pre-release version of Zulip](../production/deployment.html#installing-zulip-from-git)
using code from our [repository on GitHub](https://github.com/zulip/zulip/).
## Step 2: Install Zulip
@@ -100,6 +100,7 @@ do.
To really see Zulip in action, you'll need to get the people you work
together with using it with you.
- [Set up outgoing email](email.md) so Zulip can confirm new users'
email addresses and send notifications.
- Learn how to [get your organization started][realm-admin-docs] using
@@ -108,36 +109,38 @@ together with using it with you.
Learning more:
- Subscribe to the [Zulip announcements email
list](https://groups.google.com/forum/#!forum/zulip-announce) for
server administrators. This extremely low-traffic list is for
important announcements, including [new
releases](../overview/release-lifecycle.md) and security issues. You
can also use the [RSS
feed](https://groups.google.com/forum/#!aboutgroup/zulip-announce).
list](https://groups.google.com/forum/#!forum/zulip-announce) for
server administrators. This extremely low-traffic list is for
important announcements, including [new
releases](../overview/release-lifecycle.md) and security issues. You
can also use the [RSS
feed](https://groups.google.com/forum/#!aboutgroup/zulip-announce).
- Follow [Zulip on Twitter](https://twitter.com/zulip).
- Learn how to [configure your Zulip server settings](settings.md).
- Learn about [Backups, export and import](../production/export-and-import.md)
and [upgrading](../production/upgrade-or-modify.md) a production Zulip
server.
and [upgrading](../production/upgrade-or-modify.md) a production Zulip
server.
[realm-admin-docs]: https://zulip.com/help/getting-your-organization-started-with-zulip
```eval_rst
.. _installer-details:
```
## Details: What the installer does
The install script does several things:
- Creates the `zulip` user, which the various Zulip servers will run as.
- Creates `/home/zulip/deployments/`, which the Zulip code for this
deployment (and future deployments when you upgrade) goes into. At the
very end of the install process, the script moves the Zulip code tree
it's running from (which you unpacked from a tarball above) to a
directory there, and makes `/home/zulip/deployments/current` as a
symbolic link to it.
deployment (and future deployments when you upgrade) goes into. At the
very end of the install process, the script moves the Zulip code tree
it's running from (which you unpacked from a tarball above) to a
directory there, and makes `/home/zulip/deployments/current` as a
symbolic link to it.
- Installs Zulip's various dependencies.
- Configures the various third-party services Zulip uses, including
PostgreSQL, RabbitMQ, Memcached and Redis.
PostgreSQL, RabbitMQ, Memcached and Redis.
- Initializes Zulip's database.
If you'd like to deploy Zulip with these services on different

1
docs/production/maintain-secure-upgrade.md
View File

@@ -8,6 +8,7 @@ This was once a long page covering a bunch of topics; those topics
have since all moved to dedicated pages:
### Monitoring
Moved to [Troubleshooting](../production/troubleshooting.html#monitoring).
### Securing your Zulip server

69
docs/production/mobile-push-notifications.md
View File

@@ -11,8 +11,6 @@ will need to register your Zulip server with the Zulip mobile push
notification service. This service will forward push notifications
generated by your server to the Zulip mobile app automatically.
## How to sign up
Starting with Zulip 1.6 for both Android and iOS, Zulip servers
@@ -34,16 +32,18 @@ You can enable this for your Zulip server as follows:
need to add the line (it won't be there to uncomment).
1. If you're running Zulip 1.8.1 or newer, you can run the
registration command:
```bash
# As root:
su zulip -c '/home/zulip/deployments/current/manage.py register_server'
# Or as the zulip user, you can skip the `su zulip -c`:
/home/zulip/deployments/current/manage.py register_server
registration command:
```bash
# As root:
su zulip -c '/home/zulip/deployments/current/manage.py register_server'
# Or as the zulip user, you can skip the `su zulip -c`:
/home/zulip/deployments/current/manage.py register_server
# docker-zulip users can run this inside the container with `docker exec`:
docker exec -it -u zulip <container_name> /home/zulip/deployments/current/manage.py register_server
```
# docker-zulip users can run this inside the container with `docker exec`:
docker exec -it -u zulip <container_name> /home/zulip/deployments/current/manage.py register_server
```
This command will print the registration data it would send to the
mobile push notifications service, ask you to accept the terms of
service, and if you accept, register your server. If you have trouble,
@@ -62,15 +62,15 @@ following. Please follow the instructions carefully:
(normally they're only sent if you're idle, which isn't ideal for
this sort of testing).
- On an Android device, download and log in to the
[Zulip Android app](https://play.google.com/store/apps/details?id=com.zulipmobile).
If you were already logged in before configuring the server, you'll
need to log out first, since the app only registers for push
notifications on login.
[Zulip Android app](https://play.google.com/store/apps/details?id=com.zulipmobile).
If you were already logged in before configuring the server, you'll
need to log out first, since the app only registers for push
notifications on login.
- Hit the home button, so Zulip is running in the background, and then
have **another user** send you a **private message** (By default,
Zulip only sends push notifications for private messages sent by other
users and messages mentioning you). A push notification should appear
in the Android notification area.
have **another user** send you a **private message** (By default,
Zulip only sends push notifications for private messages sent by other
users and messages mentioning you). A push notification should appear
in the Android notification area.
[mobile-notifications-always]: https://zulip.com/help/test-mobile-notifications
@@ -119,26 +119,27 @@ and privacy in mind:
- The Push Notification Service only stores the necessary metadata for
delivering the notifications to the appropriate devices, and nothing
else:
- The APNS/FCM tokens needed to securely send mobile push
notifications to iOS and Android devices, one per device
registered to be notified by your Zulip server.
- User ID numbers generated by your Zulip server, needed to route
a given notification to the appropriate set of mobile devices.
These user ID numbers are are opaque to the Push Notification
Service and Kandra Labs.
- The APNS/FCM tokens needed to securely send mobile push
notifications to iOS and Android devices, one per device
registered to be notified by your Zulip server.
- User ID numbers generated by your Zulip server, needed to route
a given notification to the appropriate set of mobile devices.
These user ID numbers are are opaque to the Push Notification
Service and Kandra Labs.
- The Push Notification Service receives (but does not store) the
contents of individual mobile push notifications:
- The numeric message ID generated by your Zulip server.
- Metadata on the message's sender (name and avatar URL).
- Metadata on the message's recipient (stream name + ID, topic,
private message recipients, etc.).
- A timestamp.
- The message's content.
- The numeric message ID generated by your Zulip server.
- Metadata on the message's sender (name and avatar URL).
- Metadata on the message's recipient (stream name + ID, topic,
private message recipients, etc.).
- A timestamp.
- The message's content.
There's a `PUSH_NOTIFICATION_REDACT_CONTENT` setting available to
disable any message content being sent via the push notification
bouncer (i.e. message content will be replaced with
`***REDACTED***`). Note that this setting makes push notifications
`***REDACTED***`). Note that this setting makes push notifications
significantly less usable.
We plan to
@@ -146,6 +147,7 @@ and privacy in mind:
which would eliminate that usability tradeoff and additionally allow
us to not have any access to the other details mentioned in this
section.
- All of the network requests (both from Zulip servers to the Push
Notification Service and from the Push Notification Service to the
relevant Google and Apple services) are encrypted over the wire with
@@ -217,6 +219,7 @@ the app stores yourself.
If you've done that work, the Zulip server configuration for sending
push notifications through the new app is quite straightforward:
- Create a
[FCM push notifications](https://firebase.google.com/docs/cloud-messaging)
key in the Google Developer console and set `android_gcm_api_key` in

8
docs/production/password-strength.md
View File

@@ -14,7 +14,7 @@ detail how we set the default threshold (`PASSWORD_MIN_GUESSES`) we use.
First, read the doc section there. (It's short.)
Then, the CACM article ["Passwords and the Evolution of Imperfect
Authentication"][BHOS15] is comprehensive, educational, and readable,
Authentication"][bhos15] is comprehensive, educational, and readable,
and is especially recommended.
The CACM article is convincing that password requirements should be
@@ -36,7 +36,7 @@ overestimation (allowing a weak password) sharply degrades at 100k
guesses, while underestimation (rejecting a strong password) jumps up
just after 10k guesses, and grows steadily thereafter.
Moreover, the [Yahoo study][Bon12] shows that resistance to even 1M
Moreover, the [Yahoo study][bon12] shows that resistance to even 1M
guesses is more than nearly half of users accomplish with a freely
chosen password, and 100k is too much for about 20%. (See Figure 6.)
It doesn't make sense for a Zulip server to try to educate or push so
@@ -54,6 +54,6 @@ zxcvbn rarely underestimates the strength of a password too severely,
and only about 10% of users do worse than this without prompting.
[zxcvbn]: https://github.com/dropbox/zxcvbn
[BHOS15]: https://www.cl.cam.ac.uk/~fms27/papers/2015-BonneauHerOorSta-passwords.pdf
[bhos15]: https://www.cl.cam.ac.uk/~fms27/papers/2015-BonneauHerOorSta-passwords.pdf
[zxcvbn-paper]: https://www.usenix.org/system/files/conference/usenixsecurity16/sec16_paper_wheeler.pdf
[Bon12]: https://ieeexplore.ieee.org/document/6234435
[bon12]: https://ieeexplore.ieee.org/document/6234435

4
docs/production/postgresql.md
View File

@@ -1,5 +1,4 @@
PostgreSQL database details
=========================
# PostgreSQL database details
Starting with Zulip 3.0, Zulip supports a range of PostgreSQL
versions. PostgreSQL 13 is the current default for new installations;
@@ -140,7 +139,6 @@ stop PostgreSQL and restart it using pg_ctlcluster after you've debugged
with this approach, since it does bypass some of the work that
pg_ctlcluster does.
#### PostgreSQL vacuuming alerts
The `autovac_freeze` PostgreSQL alert from `check_postgres` is

30
docs/production/requirements.md
View File

@@ -1,6 +1,7 @@
# Requirements and scalability
To run a Zulip server, you will need:
- A dedicated machine or VM
- A supported OS:
- Ubuntu 20.04 Focal
@@ -44,8 +45,7 @@ sudo apt update
```
[upgrade-os]: ../production/upgrade-or-modify.html#upgrading-the-operating-system
[ubuntu-repositories]:
https://help.ubuntu.com/community/Repositories/Ubuntu
[ubuntu-repositories]: https://help.ubuntu.com/community/Repositories/Ubuntu
[enable-universe]: https://help.ubuntu.com/community/Repositories/CommandLine#Adding_the_Universe_and_Multiverse_Repositories
#### Hardware specifications
@@ -93,10 +93,10 @@ on hardware requirements for larger organizations.
- Zulip supports [running behind a reverse proxy][reverse-proxy].
- Zulip servers running inside a private network should configure the
[Smokescreen integration][smokescreen-proxy] to protect against
[SSRF attacks][SSRF], where users could make the Zulip server make
[SSRF attacks][ssrf], where users could make the Zulip server make
requests to private resources.
[SSRF]: https://owasp.org/www-community/attacks/Server_Side_Request_Forgery
[ssrf]: https://owasp.org/www-community/attacks/Server_Side_Request_Forgery
[smokescreen-proxy]: ../production/deployment.html#using-an-outgoing-http-proxy
[reverse-proxy]: ../production/deployment.html#putting-the-zulip-application-behind-a-reverse-proxy
[email-mirror-code]: https://github.com/zulip/zulip/blob/main/zerver/management/commands/email_mirror.py
@@ -140,8 +140,9 @@ Zulip in production](../production/install.md).
This section details some basic guidelines for running a Zulip server
for larger organizations (especially >1000 users or 500+ daily active
users). Zulip's resource needs depend mainly on 3 parameters:
- daily active users (e.g. number of employees if everyone's an
employee)
employee)
- total user accounts (can be much larger)
- message volume.
@@ -159,16 +160,17 @@ installing Zulip with a dedicated database server.
active users, we recommend using a [remote PostgreSQL
database](postgresql.md), but it's not required.
- **RAM:** We recommended more RAM for larger installations:
- With 25+ daily active users, 4GB of RAM.
- With 100+ daily active users, 8GB of RAM.
- With 400+ daily active users, 16GB of RAM for the Zulip
application server, plus 16GB for the database.
- With 2000+ daily active users 32GB of RAM, plus 32GB for the
database.
- Roughly linear scaling beyond that.
- **RAM:** We recommended more RAM for larger installations:
- **CPU:** The Zulip application server's CPU usage is heavily
- With 25+ daily active users, 4GB of RAM.
- With 100+ daily active users, 8GB of RAM.
- With 400+ daily active users, 16GB of RAM for the Zulip
application server, plus 16GB for the database.
- With 2000+ daily active users 32GB of RAM, plus 32GB for the
database.
- Roughly linear scaling beyond that.
- **CPU:** The Zulip application server's CPU usage is heavily
optimized due to extensive work on optimizing the performance of
requests for latency reasons. Because most servers with sufficient
RAM have sufficient CPU resources, CPU requirements are rarely an

13
docs/production/security-model.md
View File

@@ -70,10 +70,10 @@ strength allowed is controlled by two settings in
Estimating the guessability of a password is a complex problem and
impossible to efficiently do perfectly. For background or when
considering an alternate value for this setting, the article
["Passwords and the Evolution of Imperfect Authentication"][BHOS15]
["Passwords and the Evolution of Imperfect Authentication"][bhos15]
is recommended. The [2016 zxcvbn paper][zxcvbn-paper] adds useful
information about the performance of zxcvbn, and [a large 2012 study
of Yahoo users][Bon12] is informative about the strength of the
of Yahoo users][bon12] is informative about the strength of the
passwords users choose.
<!---
@@ -86,9 +86,9 @@ strength allowed is controlled by two settings in
-->
[zxcvbn]: https://github.com/dropbox/zxcvbn
[BHOS15]: http://www.cl.cam.ac.uk/~fms27/papers/2015-BonneauHerOorSta-passwords.pdf
[bhos15]: http://www.cl.cam.ac.uk/~fms27/papers/2015-BonneauHerOorSta-passwords.pdf
[zxcvbn-paper]: https://www.usenix.org/system/files/conference/usenixsecurity16/sec16_paper_wheeler.pdf
[Bon12]: http://ieeexplore.ieee.org/document/6234435/
[bon12]: http://ieeexplore.ieee.org/document/6234435/
## Messages and history
@@ -97,6 +97,7 @@ strength allowed is controlled by two settings in
attacks.
- Zulip supports both public streams and private streams.
- Any non-guest user can join any public stream in the organization,
and can view the complete message history of any public stream
without joining the stream. Guests can only access streams that
@@ -258,11 +259,11 @@ strength allowed is controlled by two settings in
the Zulip server to make HTTP requests on their behalf. As a result,
Zulip supports routing all outgoing outgoing HTTP requests [through
Smokescreen][smokescreen-setup] to ensure that Zulip cannot be
used to execute [SSRF attacks][SSRF] against other systems on an
used to execute [SSRF attacks][ssrf] against other systems on an
internal corporate network. The default Smokescreen configuration
denies access to all non-public IP addresses, including 127.0.0.1.
[SSRF]: https://owasp.org/www-community/attacks/Server_Side_Request_Forgery
[ssrf]: https://owasp.org/www-community/attacks/Server_Side_Request_Forgery
[smokescreen-setup]: ../production/deployment.html#using-an-outgoing-http-proxy
## Final notes and security response

2
docs/production/settings.md
View File

@@ -15,6 +15,7 @@ This page discusses additional configuration that a system
administrator can do. To change any of the following settings, edit
the `/etc/zulip/settings.py` file on your Zulip server, and then
restart the server with the following command:
```bash
su zulip -c '/home/zulip/deployments/current/scripts/restart-server'
```
@@ -95,6 +96,7 @@ configuration along with your other Zulip server configuration.
### Miscellaneous server settings
Some popular settings in `/etc/zulip/settings.py` include:
- The Twitter integration, which provides pretty inline previews of
tweets.
- The [email gateway](../production/email-gateway.md), which lets

13
docs/production/ssl-certificates.md
View File

@@ -11,6 +11,7 @@ chore (nor expense) that it used to be.
If you already have an SSL certificate, just install (or symlink) its
files into place at the following paths:
- `/etc/ssl/private/zulip.key` for the private key
- `/etc/ssl/certs/zulip.combined-chain.crt` for the certificate.
@@ -55,6 +56,7 @@ SSL certificates from Let's Encrypt and renew them automatically.
We recommend most Zulip servers use Certbot. You'll want something
else if:
- you have an existing workflow for managing SSL certificates
that you prefer;
- you need wildcard certificates (support from Let's Encrypt released
@@ -85,10 +87,12 @@ one as described in the section below after installing Zulip.
To enable the Certbot automation on an already-installed Zulip
server, run the following commands:
```bash
sudo -s # If not already root
/home/zulip/deployments/current/scripts/setup/setup-certbot --email=EMAIL HOSTNAME [HOSTNAME2...]
```
where HOSTNAME is the domain name users see in their browser when
using the server (e.g., `zulip.example.com`), and EMAIL is a contact
address for the server admins. Additional hostnames can also be
@@ -109,7 +113,6 @@ checks if any certificates are due for renewal, and if they are (so
approximately once every 60 days), repeats the process of request,
prove, get a fresh certificate.
## Self-signed certificate
If you aren't able to use Certbot, you can generate a self-signed SSL
@@ -126,22 +129,24 @@ just pass the `--self-signed-cert` flag when
To generate a self-signed certificate for an already-installed Zulip
server, run the following commands:
```bash
sudo -s # If not already root
/home/zulip/deployments/current/scripts/setup/generate-self-signed-cert HOSTNAME
```
where HOSTNAME is the domain name (or IP address) to use on the
generated certificate.
After replacing the certificates, you need to reload `nginx` by
running the following as `root`:
```bash
service nginx reload
```
[desktop-certs]: https://zulip.com/help/custom-certificates
## Troubleshooting
### The Android app can't connect to the server
@@ -149,7 +154,6 @@ service nginx reload
This is most often caused by an incomplete certificate chain. See
discussion in the [Manual install](#manual-install) section above.
### The iOS app can't connect to the server
This can be caused by a server set up to support only TLS 1.1 or
@@ -157,7 +161,7 @@ older (including TLS 1.0, SSL 3, or SSL 2.)
TLS 1.2 has been a standard for over 10 years, and all modern web
server software supports it. Starting in early 2020, all major
browsers [will *require* TLS 1.2 or later][tls12-required-news], and
browsers [will _require_ TLS 1.2 or later][tls12-required-news], and
will refuse to connect over TLS 1.1 or older. And on iOS, Apple [has
since iOS 9][apple-ats] required TLS 1.2 for all connections made by
apps, unless the app specifically opts into lower security.
@@ -175,7 +179,6 @@ directive][nginx-doc-protocols] in your configuration.
[nginx-doc-protocols]: https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_protocols
### The Android app connects to the server on some devices but not others
An issue on Android 7.0 ([report][android7.0-tls-issue],

4
docs/production/troubleshooting.md
View File

@@ -37,6 +37,7 @@ and restart various services.
### Checking status with `supervisorctl status`
You can check if the Zulip application is running using:
```bash
supervisorctl status
```
@@ -111,15 +112,18 @@ problems and how to resolve them:
nginx will fail to start if you configured SSL incorrectly or did
not provide SSL certificates. To fix this, configure them properly
and then run:
```bash
service nginx restart
```
- If your host is being port scanned by unauthorized users, you may see
messages in `/var/log/zulip/server.log` like
```text
2017-02-22 14:11:33,537 ERROR Invalid HTTP_HOST header: '10.2.3.4'. You may need to add u'10.2.3.4' to ALLOWED_HOSTS.
```
Django uses the hostnames configured in `ALLOWED_HOSTS` to identify
legitimate requests and block others. When an incoming request does
not have the correct HTTP Host header, Django rejects it and logs the

241
docs/production/upgrade-or-modify.md
View File

@@ -24,33 +24,34 @@ to a new Zulip release:
for all releases newer than what is currently installed.
1. Download the appropriate release tarball from
<https://www.zulip.org/dist/releases/> You can download the latest
release with:
<https://www.zulip.org/dist/releases/> You can download the latest
release with:
```bash
wget https://www.zulip.org/dist/releases/zulip-server-latest.tar.gz
```
```bash
wget https://www.zulip.org/dist/releases/zulip-server-latest.tar.gz
```
You also have the option of upgrading Zulip [to a version in a Git
repository directly](#upgrading-from-a-git-repository) or creating
your own release tarballs from a copy of the [zulip.git
repository](https://github.com/zulip/zulip) using
`tools/build-release-tarball`.
You also have the option of upgrading Zulip [to a version in a Git
repository directly](#upgrading-from-a-git-repository) or creating
your own release tarballs from a copy of the [zulip.git
repository](https://github.com/zulip/zulip) using
`tools/build-release-tarball`.
1. Log in to your Zulip and run as root:
```bash
/home/zulip/deployments/current/scripts/upgrade-zulip zulip-server-VERSION.tar.gz
```
```bash
/home/zulip/deployments/current/scripts/upgrade-zulip zulip-server-VERSION.tar.gz
```
The upgrade process will:
- Run `apt-get upgrade`
- Install new versions of Zulip's dependencies (mainly Python packages).
- (`upgrade-zulip-from-git` only) Build Zulip's frontend assets using `webpack`.
- Shut down the Zulip service
- Run a `puppet apply`
- Run any database migrations
- Bring the Zulip service back up on the new version.
The upgrade process will:
- Run `apt-get upgrade`
- Install new versions of Zulip's dependencies (mainly Python packages).
- (`upgrade-zulip-from-git` only) Build Zulip's frontend assets using `webpack`.
- Shut down the Zulip service
- Run a `puppet apply`
- Run any database migrations
- Bring the Zulip service back up on the new version.
Upgrading will result in brief downtime for the service, which should
be under 30 seconds unless there is an expensive database migration
@@ -176,6 +177,7 @@ after resolving an issue. The most common causes of errors are:
the upgrade process.
Useful logs are available in a few places:
- The Zulip upgrade scripts log all output to
`/var/log/zulip/upgrade.log`.
- The Zulip server logs all Internal Server Errors to
@@ -261,36 +263,36 @@ instructions for other supported platforms.
2. As the Zulip user, stop the Zulip server and run the following
to back up the system:
```bash
supervisorctl stop all
/home/zulip/deployments/current/manage.py backup --output=/home/zulip/release-upgrade.backup.tar.gz
```
```bash
supervisorctl stop all
/home/zulip/deployments/current/manage.py backup --output=/home/zulip/release-upgrade.backup.tar.gz
```
3. Switch to the root user and upgrade the operating system using the
OS's standard tooling. E.g. for Ubuntu, this means running
`do-release-upgrade` and following the prompts until it completes
successfully:
```bash
sudo -i # Or otherwise get a root shell
do-release-upgrade -d
```
```bash
sudo -i # Or otherwise get a root shell
do-release-upgrade -d
```
The `-d` option to `do-release-upgrade` is required because Ubuntu
20.04 is new; it will stop being necessary once the first point
release update of Ubuntu 20.04 LTS is released.
The `-d` option to `do-release-upgrade` is required because Ubuntu
20.04 is new; it will stop being necessary once the first point
release update of Ubuntu 20.04 LTS is released.
When `do-release-upgrade` asks you how to upgrade configuration
files for services that Zulip manages like Redis, PostgreSQL,
Nginx, and memcached, the best choice is `N` to keep the
currently installed version. But it's not important; the next
step will re-install Zulip's configuration in any case.
When `do-release-upgrade` asks you how to upgrade configuration
files for services that Zulip manages like Redis, PostgreSQL,
Nginx, and memcached, the best choice is `N` to keep the
currently installed version. But it's not important; the next
step will re-install Zulip's configuration in any case.
4. As root, upgrade the database to the latest version of PostgreSQL:
```bash
/home/zulip/deployments/current/scripts/setup/upgrade-postgresql
```
```bash
/home/zulip/deployments/current/scripts/setup/upgrade-postgresql
```
5. Ubuntu 20.04 has a different version of the low-level glibc
library, which affects how PostgreSQL orders text data (known as
@@ -307,11 +309,11 @@ instructions for other supported platforms.
full-text search indexes to work with the upgraded dictionary
packages:
```bash
rm -rf /srv/zulip-venv-cache/*
/home/zulip/deployments/current/scripts/lib/upgrade-zulip-stage-2 \
/home/zulip/deployments/current/ --ignore-static-assets --audit-fts-indexes
```
```bash
rm -rf /srv/zulip-venv-cache/*
/home/zulip/deployments/current/scripts/lib/upgrade-zulip-stage-2 \
/home/zulip/deployments/current/ --ignore-static-assets --audit-fts-indexes
```
This will finish by restarting your Zulip server; you should now be
able to navigate to its URL and confirm everything is working
@@ -330,27 +332,27 @@ instructions for other supported platforms.
4. As root, upgrade the database installation and OS configuration to
match the new OS version:
```bash
touch /usr/share/postgresql/10/pgroonga_setup.sql.applied
/home/zulip/deployments/current/scripts/zulip-puppet-apply -f
pg_dropcluster 10 main --stop
systemctl stop postgresql
pg_upgradecluster 9.5 main
pg_dropcluster 9.5 main
apt remove postgresql-9.5
systemctl start postgresql
systemctl restart memcached
```
```bash
touch /usr/share/postgresql/10/pgroonga_setup.sql.applied
/home/zulip/deployments/current/scripts/zulip-puppet-apply -f
pg_dropcluster 10 main --stop
systemctl stop postgresql
pg_upgradecluster 9.5 main
pg_dropcluster 9.5 main
apt remove postgresql-9.5
systemctl start postgresql
systemctl restart memcached
```
5. Finally, we need to reinstall the current version of Zulip, which
among other things will recompile Zulip's Python module
dependencies for your new version of Python:
```bash
rm -rf /srv/zulip-venv-cache/*
/home/zulip/deployments/current/scripts/lib/upgrade-zulip-stage-2 \
/home/zulip/deployments/current/ --ignore-static-assets
```
```bash
rm -rf /srv/zulip-venv-cache/*
/home/zulip/deployments/current/scripts/lib/upgrade-zulip-stage-2 \
/home/zulip/deployments/current/ --ignore-static-assets
```
This will finish by restarting your Zulip server; you should now
be able to navigate to its URL and confirm everything is working
@@ -361,9 +363,9 @@ instructions for other supported platforms.
7. As root, finish by verifying the contents of the full-text indexes:
```bash
/home/zulip/deployments/current/manage.py audit_fts_indexes
```
```bash
/home/zulip/deployments/current/manage.py audit_fts_indexes
```
### Upgrading from Ubuntu 14.04 Trusty to 16.04 Xenial
@@ -378,27 +380,27 @@ instructions for other supported platforms.
4. As root, upgrade the database installation and OS configuration to
match the new OS version:
```bash
apt remove upstart -y
/home/zulip/deployments/current/scripts/zulip-puppet-apply -f
pg_dropcluster 9.5 main --stop
systemctl stop postgresql
pg_upgradecluster -m upgrade 9.3 main
pg_dropcluster 9.3 main
apt remove postgresql-9.3
systemctl start postgresql
service memcached restart
```
```bash
apt remove upstart -y
/home/zulip/deployments/current/scripts/zulip-puppet-apply -f
pg_dropcluster 9.5 main --stop
systemctl stop postgresql
pg_upgradecluster -m upgrade 9.3 main
pg_dropcluster 9.3 main
apt remove postgresql-9.3
systemctl start postgresql
service memcached restart
```
5. Finally, we need to reinstall the current version of Zulip, which
among other things will recompile Zulip's Python module
dependencies for your new version of Python:
```bash
rm -rf /srv/zulip-venv-cache/*
/home/zulip/deployments/current/scripts/lib/upgrade-zulip-stage-2 \
/home/zulip/deployments/current/ --ignore-static-assets
```
```bash
rm -rf /srv/zulip-venv-cache/*
/home/zulip/deployments/current/scripts/lib/upgrade-zulip-stage-2 \
/home/zulip/deployments/current/ --ignore-static-assets
```
This will finish by restarting your Zulip server; you should now be
able to navigate to its URL and confirm everything is working
@@ -429,27 +431,27 @@ instructions for other supported platforms.
4. As root, upgrade the database installation and OS configuration to
match the new OS version:
```bash
apt remove upstart -y
/home/zulip/deployments/current/scripts/zulip-puppet-apply -f
pg_dropcluster 11 main --stop
systemctl stop postgresql
pg_upgradecluster -m upgrade 9.6 main
pg_dropcluster 9.6 main
apt remove postgresql-9.6
systemctl start postgresql
service memcached restart
```
```bash
apt remove upstart -y
/home/zulip/deployments/current/scripts/zulip-puppet-apply -f
pg_dropcluster 11 main --stop
systemctl stop postgresql
pg_upgradecluster -m upgrade 9.6 main
pg_dropcluster 9.6 main
apt remove postgresql-9.6
systemctl start postgresql
service memcached restart
```
5. Finally, we need to reinstall the current version of Zulip, which
among other things will recompile Zulip's Python module
dependencies for your new version of Python:
```bash
rm -rf /srv/zulip-venv-cache/*
/home/zulip/deployments/current/scripts/lib/upgrade-zulip-stage-2 \
/home/zulip/deployments/current/ --ignore-static-assets
```
```bash
rm -rf /srv/zulip-venv-cache/*
/home/zulip/deployments/current/scripts/lib/upgrade-zulip-stage-2 \
/home/zulip/deployments/current/ --ignore-static-assets
```
This will finish by restarting your Zulip server; you should now
be able to navigate to its URL and confirm everything is working
@@ -469,9 +471,9 @@ instructions for other supported platforms.
8. As root, finish by verifying the contents of the full-text indexes:
```bash
/home/zulip/deployments/current/manage.py audit_fts_indexes
```
```bash
/home/zulip/deployments/current/manage.py audit_fts_indexes
```
## Upgrading PostgreSQL
@@ -487,34 +489,33 @@ To upgrade the version of PostgreSQL on the Zulip server:
1. Stop the server, as the `zulip` user:
```bash
# On Zulip before 4.0, use `supervisor stop all` instead
/home/zulip/deployments/current/scripts/stop-server
```
```bash
# On Zulip before 4.0, use `supervisor stop all` instead
/home/zulip/deployments/current/scripts/stop-server
```
1. Take a backup, in case of any problems:
```bash
/home/zulip/deployments/current/manage.py backup --output=/home/zulip/postgresql-upgrade.backup.tar.gz
```
```bash
/home/zulip/deployments/current/manage.py backup --output=/home/zulip/postgresql-upgrade.backup.tar.gz
```
1. As root, run the database upgrade tool:
```bash
/home/zulip/deployments/current/scripts/setup/upgrade-postgresql
```
```bash
/home/zulip/deployments/current/scripts/setup/upgrade-postgresql
```
1. As the `zulip` user, start the server again:
```bash
# On Zulip before 4.0, use `restart-server` instead of `start-server` instead
/home/zulip/deployments/current/scripts/start-server
```
```bash
# On Zulip before 4.0, use `restart-server` instead of `start-server` instead
/home/zulip/deployments/current/scripts/start-server
```
You should now be able to navigate to the Zulip server's URL and
confirm everything is working correctly.
## Modifying Zulip
Zulip is 100% free and open source software, and you're welcome to
@@ -526,10 +527,10 @@ modified version of Zulip, please be responsible about communicating
that fact:
- Ideally, you'd reproduce the issue in an unmodified version (e.g. on
[chat.zulip.org](../contributing/chat-zulip-org.md) or
[zulip.com](https://zulip.com)).
[chat.zulip.org](../contributing/chat-zulip-org.md) or
[zulip.com](https://zulip.com)).
- Where that is difficult or you think it's very unlikely your changes
are related to the issue, just mention your changes in the issue report.
are related to the issue, just mention your changes in the issue report.
If you're looking to modify Zulip by applying changes developed by the
Zulip core team and merged into `main`, skip to [this
@@ -589,9 +590,9 @@ git push origin +acme-branch
```
- Log in to your Zulip server and configure and use
[upgrade-zulip-from-git][] to install the changes; remember to
configure `git_repo_url` to point to your fork on GitHub and run it as
`upgrade-zulip-from-git acme-branch`.
[upgrade-zulip-from-git][] to install the changes; remember to
configure `git_repo_url` to point to your fork on GitHub and run it as
`upgrade-zulip-from-git acme-branch`.
This workflow solves all of the problems described above: your change
will be compiled and installed correctly (restarting the server), and

26
docs/production/upload-backends.md
View File

@@ -21,10 +21,10 @@ Here, we document the process for configuring Zulip's S3 file upload
backend. To enable this backend, you need to do the following:
1. In the AWS management console, create a new IAM account (aka API
user) for your Zulip server, and two buckets in S3, one for uploaded
files included in messages, and another for user avatars. You need
two buckets because the "user avatars" bucket is generally configured
as world-readable, whereas the "uploaded files" one is not.
user) for your Zulip server, and two buckets in S3, one for uploaded
files included in messages, and another for user avatars. You need
two buckets because the "user avatars" bucket is generally configured
as world-readable, whereas the "uploaded files" one is not.
1. Set `s3_key` and `s3_secret_key` in /etc/zulip/zulip-secrets.conf
to be the S3 access and secret keys for the IAM account.
@@ -140,18 +140,18 @@ your local backend to Amazon S3. Follow these instructions, step by
step, to do the migration.
1. First, [set up the S3 backend](#s3-backend-configuration) in the settings
(all the auth stuff), but leave `LOCAL_UPLOADS_DIR` set -- the
migration tool will need that value to know where to find your uploads.
(all the auth stuff), but leave `LOCAL_UPLOADS_DIR` set -- the
migration tool will need that value to know where to find your uploads.
2. Run `./manage.py transfer_uploads_to_s3`. This will upload all the
files from the local uploads directory to Amazon S3. By default,
this command runs on 6 parallel processes, since uploading is a
latency-sensitive operation. You can control this parameter using
the `--processes` option.
files from the local uploads directory to Amazon S3. By default,
this command runs on 6 parallel processes, since uploading is a
latency-sensitive operation. You can control this parameter using
the `--processes` option.
3. Once the transfer script completes, disable `LOCAL_UPLOADS_DIR`, and
restart your server (continuing the last few steps of the S3
backend setup instructions).
restart your server (continuing the last few steps of the S3
backend setup instructions).
Congratulations! Your uploaded files are now migrated to S3.
**Caveat**: The current version of this tool does not migrate an
uploaded organization avatar or logo.
uploaded organization avatar or logo.

6
docs/subsystems/analytics.md
View File

@@ -29,7 +29,7 @@ There are three main components:
The next several sections will dive into the details of these components.
## The *Count database tables
## The \*Count database tables
The Zulip analytics system is built around collecting time series data in a
set of database tables. Each of these tables has the following fields:
@@ -76,7 +76,7 @@ by the system and with what data.
## The FillState table
The default Zulip production configuration runs a cron job once an hour that
updates the *Count tables for each of the CountStats in the COUNT_STATS
updates the \*Count tables for each of the CountStats in the COUNT_STATS
dictionary. The FillState table simply keeps track of the last end_time that
we successfully updated each stat. It also enables the analytics system to
recover from errors (by retrying) and to monitor that the cron job is
@@ -103,7 +103,7 @@ There are a few important principles that we use to make the system
efficient:
- Not repeating work to keep things up to date (via FillState)
- Storing data in the *Count tables to avoid our endpoints hitting the core
- Storing data in the \*Count tables to avoid our endpoints hitting the core
Message/UserMessage tables is key, because some queries could take minutes
to calculate. This allows any expensive operations to run offline, and
then the endpoints to server data to users can be fast.

2
docs/subsystems/api-release-checklist.md
View File

@@ -12,7 +12,7 @@ The steps below assume that you are familiar with the material
presented [here](https://packaging.python.org/tutorials/installing-packages/).
1. [Reconfigure the package][2], if need be (upgrade version
number, development status, and so on).
number, development status, and so on).
2. Create a [source distribution][3].

2
docs/subsystems/billing.md
View File

@@ -4,6 +4,7 @@ Zulip uses a third party (Stripe) for billing, so working on the billing
system requires a little bit of setup.
To set up the development environment to work on the billing code:
- Create a Stripe account
- Go to <https://dashboard.stripe.com/account/apikeys>, and add the
publishable key and secret key as `stripe_publishable_key` and
@@ -15,6 +16,7 @@ Nearly all the billing-relevant code lives in `corporate/`.
Stripe makes pretty regular updates to their API. The process for upgrading
our code is:
- Go to <https://dashboard.stripe.com/developers> in your Stripe account.
- Upgrade the API version.
- Run `tools/test-backend --generate-stripe-fixtures`

7
docs/subsystems/caching.md
View File

@@ -59,7 +59,7 @@ The `get_user` function is a pretty typical piece of code using this
framework; as you can see, it's very little code on top of our
`cache_with_key` decorator:
``` python
```python
def user_profile_cache_key_id(email: str, realm_id: int) -> str:
return u"user_profile:%s:%s" % (make_safe_digest(email.strip()), realm_id,)
@@ -73,6 +73,7 @@ def get_user(email: str, realm: Realm) -> UserProfile:
```
This decorator implements a pretty classic caching paradigm:
- The `user_profile_cache_key` function defines a unique map from a
canonical form of its arguments to a string. These strings are
namespaced (the `user_profile:` part) so that they won't overlap
@@ -237,6 +238,7 @@ objects to minimize data transfer between Django and memcached).
We generally try to avoid in-process backend caching in Zulip's Django
codebase, because every Zulip production installation involves
multiple servers. We do have a few, however:
- `per_request_display_recipient_cache`: A cache flushed at the start
of every request; this simplifies correctly implementing our goal of
not repeatedly fetching the "display recipient" (e.g. stream name)
@@ -260,5 +262,4 @@ cached by clients is changed. Clients are responsible for handling
the events, updating their state, and rerendering any UI components
that might display the modified state.
[post-save-signals]:
https://docs.djangoproject.com/en/2.0/ref/signals/#post-save
[post-save-signals]: https://docs.djangoproject.com/en/2.0/ref/signals/#post-save

11
docs/subsystems/dependencies.md
View File

@@ -103,6 +103,7 @@ on specific versions of these packages wherever possible.
The exact lists of `apt` packages needed by Zulip are maintained in a
few places:
- For production, in our Puppet configuration, `puppet/zulip/`, using
the `Package` and `SafePackage` directives.
- For development, in `SYSTEM_DEPENDENCIES` in `tools/lib/provision.py`.
@@ -249,12 +250,12 @@ installer for `yarn`, modified to support installing to a path that is
not the current user's home directory).
- `nvm` has its own system for installing each version of `node` at
its own path, which we use, though we install a `/usr/local/bin/node`
wrapper to access the desired version conveniently and efficiently
(`nvm` has a lot of startup overhead).
its own path, which we use, though we install a `/usr/local/bin/node`
wrapper to access the desired version conveniently and efficiently
(`nvm` has a lot of startup overhead).
- `install-yarn.sh` is configured to install `yarn` at
`/srv/zulip-yarn`. We don't do anything special to try to manage
multiple versions of `yarn`.
`/srv/zulip-yarn`. We don't do anything special to try to manage
multiple versions of `yarn`.
## Other third-party and generated files

1
docs/subsystems/email.md
View File

@@ -32,6 +32,7 @@ with only a few things you need to know to get started.
One slightly complicated decision you may have to make when adding an email
is figuring out how to schedule it. There are 3 ways to schedule email.
- Send it immediately, in the current Django process, e.g. by calling
`send_email` directly. An example of this is the `confirm_registration`
email.

1
docs/subsystems/emoji.md
View File

@@ -68,6 +68,7 @@ emoji tooling). See [our dependencies document](../subsystems/dependencies.md)
for more details on this strategy.
The emoji tree generated by this process contains several import elements:
- `emoji_codes.json`: A set of mappings used by the Zulip frontend to
understand what Unicode emoji exist and what their shortnames are,
used for autocomplete, emoji pickers, etc. This has been

10
docs/subsystems/full-text-search.md
View File

@@ -63,8 +63,8 @@ All steps in this section should be run as the `root` user; on most installs, th
su zulip -c '/home/zulip/deployments/current/manage.py migrate pgroonga'
```
Note that the migration may take a long time, and users will be
unable to send new messages until the migration finishes.
Note that the migration may take a long time, and users will be
unable to send new messages until the migration finishes.
1. Once the migrations are complete, restart Zulip:
@@ -80,9 +80,9 @@ All steps in this section should be run as the `root` user; on most installs, th
su zulip -c '/home/zulip/deployments/current/manage.py migrate pgroonga zero'
```
If you intend to re-enable PGroonga later, you can skip this step,
at the cost of your Message table being slightly larger than it would
be otherwise.
If you intend to re-enable PGroonga later, you can skip this step,
at the cost of your Message table being slightly larger than it would
be otherwise.
1. Edit `/etc/zulip/settings.py`, editing the line containing `USING_PGROONGA` to read:

4
docs/subsystems/hashchange-system.md
View File

@@ -14,7 +14,7 @@ Some examples are:
"announce") selected.
- `/#narrow/stream/42-android/topic/fun`: Message feed showing stream
"android" and topic "fun". (The `42` represents the id of the
stream.
stream.
The main module in the frontend that manages this all is
`static/js/hashchange.js` (plus `hash_util.js` for all the parsing
@@ -28,7 +28,7 @@ different flows:
`/#streams`. This makes it easy to have simple links around the app
without custom click handlers for each one.
- The user uses the "back" button in their browser (basically
equivalent to the previous one, as a *link* out of the browser history
equivalent to the previous one, as a _link_ out of the browser history
will be visited).
- The user clicking some in-app click handler (e.g. "Stream settings"
for an individual stream), that potentially does

6
docs/subsystems/hotspots.md
View File

@@ -32,8 +32,8 @@ ALL_HOTSPOTS = {
The target element and visual orientation of each hotspot is specified in
`HOTSPOT_LOCATIONS` of `static/js/hotspots.js`.
The `icon_offset` property specifies where the pulsing icon is placed *relative to
the width and height of the target element*.
The `icon_offset` property specifies where the pulsing icon is placed _relative to
the width and height of the target element_.
By default, `popovers.compute_placement` is used to responsively
determine whether a popover is best displayed above (TOP), below (BOTTOM),
@@ -53,6 +53,7 @@ in multiple copies of hotspots appearing; you can clear that by
reloading the browser.
Here are some visual characteristics to confirm:
- popover content is readable
- icons reposition themselves on resize
- icons are hidden and shown along with their associated elements
@@ -67,6 +68,7 @@ a target element on a sidebar or overlay, the icon's z-index may need to
be increased to 101, 102, or 103.
This adjustment can be made at the bottom of `static/styles/hotspots.css`:
```css
#hotspot_new_hotspot_name_icon {
z-index: 103;

31
docs/subsystems/html-css.md
View File

@@ -63,15 +63,15 @@ browsers to make sure things look the same.
### Behavior
- Templates are automatically recompiled in development when the file
is saved; a refresh of the page should be enough to display the latest
version. You might need to do a hard refresh, as some browsers cache
webpages.
is saved; a refresh of the page should be enough to display the latest
version. You might need to do a hard refresh, as some browsers cache
webpages.
- Variables can be used in templates. The variables available to the
template are called the **context**. Passing the context to the HTML
template sets the values of those variables to the value they were
given in the context. The sections below contain specifics on how the
context is defined and where it can be found.
template are called the **context**. Passing the context to the HTML
template sets the values of those variables to the value they were
given in the context. The sections below contain specifics on how the
context is defined and where it can be found.
### Backend templates
@@ -85,11 +85,11 @@ found [here][jconditionals].
The context for Jinja2 templates is assembled from a few places:
- `zulip_default_context` in `zerver/context_processors.py`. This is
the default context available to all Jinja2 templates.
the default context available to all Jinja2 templates.
- As an argument in the `render` call in the relevant function that
renders the template. For example, if you want to find the context
passed to `index.html`, you can do:
renders the template. For example, if you want to find the context
passed to `index.html`, you can do:
```console
$ git grep zerver/app/index.html '*.py'
@@ -99,7 +99,7 @@ zerver/views/home.py: response = render(request, 'zerver/app/index.html',
The next line in the code being the context definition.
- `zproject/urls.py` for some fairly static pages that are rendered
using `TemplateView`, for example:
using `TemplateView`, for example:
```python
path('config-error/google', TemplateView.as_view(
@@ -224,9 +224,10 @@ If you want to test minified files in development, look for the
### How it works in production
A few useful notes are:
- Zulip installs static assets in production in
`/home/zulip/prod-static`. When a new version is deployed, before the
server is restarted, files are copied into that directory.
`/home/zulip/prod-static`. When a new version is deployed, before the
server is restarted, files are copied into that directory.
- We use the VFL (versioned file layout) strategy, where each file in
the codebase (e.g. `favicon.ico`) gets a new name
(e.g. `favicon.c55d45ae8c58.ico`) that contains a hash in it. Each
@@ -281,8 +282,8 @@ production browser console. If you need to access a variable or
function in those scenarios, add it to `zulip_test`. This is also
**not** a stable API.
[Jinja2]: http://jinja.pocoo.org/
[Handlebars]: https://handlebarsjs.com/
[jinja2]: http://jinja.pocoo.org/
[handlebars]: https://handlebarsjs.com/
[trans]: http://jinja.pocoo.org/docs/dev/templates/#i18n
[jconditionals]: http://jinja.pocoo.org/docs/2.9/templates/#list-of-control-structures
[hconditionals]: https://handlebarsjs.com/guide/#block_helpers.html

1
docs/subsystems/input-pills.md
View File

@@ -66,7 +66,6 @@ export function filter_taken_users(items, pill_widget) {
You can get notifications from the pill code that pills have been
created/remove.
```js
pills.onPillCreate(function () {
update_save_state();

8
docs/subsystems/logging.md
View File

@@ -84,6 +84,7 @@ can be extremely valuable for investigating performance problems.
```
The format of this output is:
- Timestamp
- Log level
- Logger name, abbreviated as "zr" for these Zulip request logs
@@ -92,11 +93,11 @@ The format of this output is:
- HTTP status code
- Time to process
- (Optional perf data details, e.g. database time/queries, memcached
time/queries, Django process startup time, Markdown processing time,
etc.)
time/queries, Django process startup time, Markdown processing time,
etc.)
- Endpoint/URL from zproject/urls.py
- "email via client" showing user account involved (if logged in) and
the type of client they used ("web", "Android", etc.).
the type of client they used ("web", "Android", etc.).
The performance data details are particularly useful for investigating
performance problems, since one can see at a glance whether a slow
@@ -136,6 +137,7 @@ new feature hard to miss.
`blueslip = require("./static/js/blueslip"); blueslip.get_log()`.
Blueslip supports several error levels:
- `throw new Error(…)`: For fatal errors that cannot be easily
recovered from. We try to avoid using it, since it kills the
current JS thread, rather than returning execution to the caller.

2
docs/subsystems/management-commands.md
View File

@@ -21,7 +21,7 @@ written for a range of purposes:
- The actual scripts run by supervisord to run the persistent
processes in a Zulip server, e.g. `runtornado` and `process_queue`.
- For a sysadmin to verify a Zulip server's configuration during
installation, e.g. `checkconfig`, `send_test_email`.
installation, e.g. `checkconfig`, `send_test_email`.
- As the interface for doing those rare operations that don't have a
UI yet, e.g. `deactivate_realm`, `reactivate_realm`,
`change_user_email` (for the case where the user doesn't control the

1
docs/subsystems/markdown.md
View File

@@ -255,7 +255,6 @@ accurate.
- Enable linking to other streams using `#**streamName**`.
### Code
- Enable fenced code block extension, with syntax highlighting.

59
docs/subsystems/notifications.md
View File

@@ -25,24 +25,25 @@ sort of notifications system:
As a reminder, the relevant part of the flow for sending messages is
as follows:
- `do_send_messages` is the synchronous message-sending code path,
and passing the following data in its `send_event` call:
- Data about the message's content (E.g. mentions, wildcard
mentions, and alert words) and encodes it into the `UserMessage`
table's `flags` structure, which is in turn passed into
`send_event` for each user receiving the message.
mentions, and alert words) and encodes it into the `UserMessage`
table's `flags` structure, which is in turn passed into
`send_event` for each user receiving the message.
- Data about user configuration relevant to the message, such as
`push_notify_user_ids` and `stream_notify_user_ids`, are included
alongside `flags` in the per-user data structure.
`push_notify_user_ids` and `stream_notify_user_ids`, are included
alongside `flags` in the per-user data structure.
- The `presence_idle_user_ids` set, containing the subset of
recipient users who are mentioned, are PM recipients, have alert
words, or otherwise would normally get a notification, but have not
interacted with a Zulip client in the last few minutes. (Users who
have generally will not receive a notification unless the
`enable_online_push_notifications` flag is enabled). This data
structure ignores users for whom the message is not notifiable,
which is important to avoid this being thousands of `user_ids` for
messages to large streams with few currently active users.
recipient users who are mentioned, are PM recipients, have alert
words, or otherwise would normally get a notification, but have not
interacted with a Zulip client in the last few minutes. (Users who
have generally will not receive a notification unless the
`enable_online_push_notifications` flag is enabled). This data
structure ignores users for whom the message is not notifiable,
which is important to avoid this being thousands of `user_ids` for
messages to large streams with few currently active users.
- The Tornado [event queue system](../subsystems/events-system.md)
processes that data, as well as data about each user's active event
queues, to (1) push an event to each queue needing that message and
@@ -110,23 +111,23 @@ as follows:
disabled are rechecked, as the user may have disabled one of these
settings during the queuing period.
- The **Email notifications queue processor**, `MissedMessageWorker`,
takes care to wait for 2 minutes (hopefully in the future this will be a
configuration setting) and starts a thread to batch together multiple
messages into a single email. These features are unnecessary
for mobile push notifications, because we can live-update those
details with a future notification, whereas emails cannot be readily
updated once sent. Zulip's email notifications are styled similarly
to GitHub's email notifications, with a clean, simple design that
makes replying from an email client possible (using the [incoming
email integration](../production/email-gateway.md)).
takes care to wait for 2 minutes (hopefully in the future this will be a
configuration setting) and starts a thread to batch together multiple
messages into a single email. These features are unnecessary
for mobile push notifications, because we can live-update those
details with a future notification, whereas emails cannot be readily
updated once sent. Zulip's email notifications are styled similarly
to GitHub's email notifications, with a clean, simple design that
makes replying from an email client possible (using the [incoming
email integration](../production/email-gateway.md)).
- The **Push notifications queue processor**,
`PushNotificationsWorker`, is a simple wrapper around the
`push_notifications.py` code that actually sends the
notification. This logic is somewhat complicated by having to track
the number of unread push notifications to display on the mobile
apps' badges, as well as using the [mobile push notifications
service](../production/mobile-push-notifications.md) for self-hosted
systems.
`PushNotificationsWorker`, is a simple wrapper around the
`push_notifications.py` code that actually sends the
notification. This logic is somewhat complicated by having to track
the number of unread push notifications to display on the mobile
apps' badges, as well as using the [mobile push notifications
service](../production/mobile-push-notifications.md) for self-hosted
systems.
The following important constraints are worth understanding about the
structure of the system, when thinking about changes to it:

5
docs/subsystems/performance.md
View File

@@ -29,6 +29,7 @@ important to understand the load profiles for production uses.
Zulip servers typically involve a mixture of two very different types
of load profiles:
- Open communities like open source projects, online classes,
etc. have large numbers of users, many of whom are idle. (Many of
the others likely stopped by to ask a question, got it answered, and
@@ -83,7 +84,7 @@ substantial oscillation within a 24 hour period), we expect the rough
sense of them (as well as the list of important endpoints) is not
likely to vary dramatically over time.
``` eval_rst
```eval_rst
======================= ============ ============== ===============
Endpoint Average time Request volume Average impact
======================= ============ ============== ===============
@@ -181,7 +182,7 @@ Zulip is somewhat unusual among webapps in sending essentially all of the
data required for the entire Zulip webapp in this single request,
which is part of why the Zulip webapp loads very quickly -- one only
needs a single round trip aside from cacheable assets (avatars, images, JS,
CSS). Data on other users in the organization, streams, supported
CSS). Data on other users in the organization, streams, supported
emoji, custom profile fields, etc., is all included. The nice thing
about this model is that essentially every UI element in the Zulip
client can be rendered immediately without paying latency to the

2
docs/subsystems/pointer.md
View File

@@ -28,7 +28,7 @@ First a bit of terminology:
### Recipient bar: message you clicked
If you enter a narrow by clicking on a message group's *recipient bar*
If you enter a narrow by clicking on a message group's _recipient bar_
(stream/topic or private message recipient list at the top of a group
of messages), Zulip will select the message you clicked on. This
provides a nice user experience where you get to see the stuff near

6
docs/subsystems/realms.md
View File

@@ -1,6 +1,6 @@
# Realms in Zulip
Zulip allows multiple *realms* to be hosted on a single instance.
Zulip allows multiple _realms_ to be hosted on a single instance.
Realms are the Zulip codebases's internal name for what we refer to in
user documentation as an organization (the name "realm" comes from
[Kerberos](https://web.mit.edu/kerberos/)).
@@ -66,7 +66,7 @@ user-facing documentation on this.
By default, Linux does not provide a convenient way to use subdomains
in your local development environment. To solve this problem, we use
the **zulipdev.com** domain, which has a wildcard A record pointing to
127.0.0.1. You can use zulipdev.com to connect to your Zulip
127.0.0.1. You can use zulipdev.com to connect to your Zulip
development server instead of localhost. The default realm with the
Shakespeare users has the subdomain `zulip` and can be accessed by
visiting **zulip.zulipdev.com**.
@@ -77,7 +77,7 @@ will try to get the page on your behalf. Since zulipdev.com points
to 127.0.0.1 the proxy server is likely to give you a 503 error. The
workaround is to disable your proxy for `*.zulipdev.com`. The DNS
lookup should still work even if you disable proxy for
*.zulipdev.com. If it doesn't you can add zulipdev.com records in
\*.zulipdev.com. If it doesn't you can add zulipdev.com records in
`/etc/hosts` file. The file should look something like this.
```text

1
docs/subsystems/release-checklist.md
View File

@@ -65,6 +65,7 @@ preparing a new release.
**Note:** This will trigger the [GitHub action](https://github.com/zulip/zulip/blob/main/tools/oneclickapps/README.md)
for updating DigitalOcean one-click app image. The action uses the latest release
tarball published on `zulip.org` for creating the image.
- Update the [Docker image](https://github.com/zulip/docker-zulip) and
do a release of that.
- Update the image of DigitalOcean one click app using

12
docs/subsystems/schema-migrations.md
View File

@@ -93,16 +93,18 @@ migrations.
Another important note is that making changes to the data in a table
via `RunPython` code and `ALTER TABLE` operations within a single,
atomic migration don't mix well. If you encounter an error such as
```text
django.db.utils.OperationalError: cannot ALTER TABLE "table_name" because it has pending trigger events
```
when testing the migration, the reason is often that these operations
were incorrectly mixed. To resolve this, consider making the migration
non-atomic, splitting it into two migration files (recommended), or replacing the
`RunPython` logic with pure SQL (though this can generally be difficult).
- **Making large migrations work**. Major migrations should have a
few properties:
few properties:
- **Unit tests**. You'll want to carefully test these, so you might
as well write some unit tests to verify the migration works
@@ -120,13 +122,13 @@ few properties:
the migration can even continue where it left off, without needing
to redo work.
- **Multi-step migrations**. For really big migrations, one wants
to split the transition into into several commits that are each
individually correct, and can each be deployed independently:
to split the transition into into several commits that are each
individually correct, and can each be deployed independently:
1. First, do a migration to add the new column to the Message table
and start writing to that column (but don't use it for anything)
and start writing to that column (but don't use it for anything)
2. Second, do a migration to copy values from the old column to
the new column, to ensure that the two data stores agree.
the new column, to ensure that the two data stores agree.
3. Third, a commit that stops writing to the old field.
4. Any cleanup work, e.g. if the old field were a column, we'd do
a migration to remove it entirely here.

171
docs/subsystems/sending-messages.md
View File

@@ -17,16 +17,17 @@ and we generally don't repeat the content discussed there.
This is just a bit of terminology: A "message list" is what Zulip
calls the frontend concept of a (potentially narrowed) message feed.
There are 3 related structures:
- A `message_list_data` just has the sequencing data of which message
IDs go in what order.
IDs go in what order.
- A `message_list` is built on top of `message_list_data` and
additionally contains the data for a visible-to-the-user message list
(E.g. where trailing bookends should appear, a selected message,
etc.).
additionally contains the data for a visible-to-the-user message list
(E.g. where trailing bookends should appear, a selected message,
etc.).
- A `message_list_view` is built on top of `message_list` and
additionally contains rendering details like a window of up to 400
messages that is present in the DOM at the time, scroll position
controls, etc.
additionally contains rendering details like a window of up to 400
messages that is present in the DOM at the time, scroll position
controls, etc.
(This should later be expanded into a full article on message lists
and narrowing).
@@ -49,26 +50,26 @@ process described in our
This section details the ways in which it is different:
- There is significant custom code inside the `process_message_event`
function in `zerver/tornado/event_queue.py`. This custom code has a
number of purposes:
- Triggering [email and mobile push
notifications](../subsystems/notifications.md) for any users who
do not have active clients and have settings of the form "push
notifications when offline". In order to avoid doing any real
computational work inside the Tornado codebase, this logic aims
to just do the check for whether a notification should be
generated, and then put an event into an appropriate
[queue](../subsystems/queuing.md) to actually send the message.
See `maybe_enqueue_notifications` and related code for this part
of the logic.
- Splicing user-dependent data (E.g. `flags` such as when the user
was `mentioned`) into the events.
- Handling the [local echo details](#local-echo).
- Handling certain client configuration options that affect
messages. E.g. determining whether to send the
plaintext/Markdown raw content or the rendered HTML (e.g. the
`apply_markdown` and `client_gravatar` features in our
[events API docs](https://zulip.com/api/register-queue)).
function in `zerver/tornado/event_queue.py`. This custom code has a
number of purposes:
- Triggering [email and mobile push
notifications](../subsystems/notifications.md) for any users who
do not have active clients and have settings of the form "push
notifications when offline". In order to avoid doing any real
computational work inside the Tornado codebase, this logic aims
to just do the check for whether a notification should be
generated, and then put an event into an appropriate
[queue](../subsystems/queuing.md) to actually send the message.
See `maybe_enqueue_notifications` and related code for this part
of the logic.
- Splicing user-dependent data (E.g. `flags` such as when the user
was `mentioned`) into the events.
- Handling the [local echo details](#local-echo).
- Handling certain client configuration options that affect
messages. E.g. determining whether to send the
plaintext/Markdown raw content or the rendered HTML (e.g. the
`apply_markdown` and `client_gravatar` features in our
[events API docs](https://zulip.com/api/register-queue)).
- Following our standard naming convention, input validation is done
inside the `check_message` function in `zerver/lib/actions.py`, which is responsible for
validating the user can send to the recipient,
@@ -78,27 +79,27 @@ number of purposes:
the message) in `zerver/lib/actions.py` is one of the most optimized and thus complex parts of
the system. But in short, its job is to atomically do a few key
things:
- Store a `Message` row in the database.
- Store one `UserMessage` row in the database for each user who is
a recipient of the message (including the sender), with
appropriate `flags` for whether the user was mentioned, an alert
word appears, etc. See
[the section on soft deactivation](#soft-deactivation) for
a clever optimization we use here that is important for large
open organizations.
- Do all the database queries to fetch relevant data for and then
send a `message` event to the
[events system](../subsystems/events-system.md) containing the
data it will need for the calculations described above. This
step adds a lot of complexity, because the events system cannot
make queries to the database directly.
- Trigger any other deferred work caused by the current message,
e.g. [outgoing webhooks](https://zulip.com/api/outgoing-webhooks)
or embedded bots.
- Every query is designed to be a bulk query; we carefully
unit-test this system for how many database and memcached queries
it makes when sending messages with large numbers of recipients,
to ensure its performance.
- Store a `Message` row in the database.
- Store one `UserMessage` row in the database for each user who is
a recipient of the message (including the sender), with
appropriate `flags` for whether the user was mentioned, an alert
word appears, etc. See
[the section on soft deactivation](#soft-deactivation) for
a clever optimization we use here that is important for large
open organizations.
- Do all the database queries to fetch relevant data for and then
send a `message` event to the
[events system](../subsystems/events-system.md) containing the
data it will need for the calculations described above. This
step adds a lot of complexity, because the events system cannot
make queries to the database directly.
- Trigger any other deferred work caused by the current message,
e.g. [outgoing webhooks](https://zulip.com/api/outgoing-webhooks)
or embedded bots.
- Every query is designed to be a bulk query; we carefully
unit-test this system for how many database and memcached queries
it makes when sending messages with large numbers of recipients,
to ensure its performance.
## Local echo
@@ -176,6 +177,7 @@ implementation was under 150 lines of code.
This section just has a brief review of the sequence of steps all in
one place:
- User hits send in the compose box.
- Compose box validation runs; if it passes, the browser locally
echoes the message and then sends a request to the `POST /messages`
@@ -183,7 +185,7 @@ one place:
- The Django URL routes and middleware run, and eventually call the
`send_message_backend` view function in `zerver/views/messages.py`.
(Alternatively, for an API request to send a message via Zulip's
REST API, things start here).
REST API, things start here).
- `send_message_backend` does some validation before triggering the
`check_message` + `do_send_messages` backend flow.
- That backend flow saves the data to the database and triggers a
@@ -233,17 +235,17 @@ significant delay in rendering the message and delivering it to other
users.
- For this case, Zulip's backend Markdown processor will render the
message without including the URL embeds/previews, but it will add a
deferred work item into the `embed_links` queue.
message without including the URL embeds/previews, but it will add a
deferred work item into the `embed_links` queue.
- The [queue processor](../subsystems/queuing.md) for the
`embed_links` queue will fetch the URLs, and then if they return
results, rerun the Markdown processor and notify clients of the
updated message `rendered_content`.
`embed_links` queue will fetch the URLs, and then if they return
results, rerun the Markdown processor and notify clients of the
updated message `rendered_content`.
- We reuse the `update_message` framework (used for
Zulip's message editing feature) in order to avoid needing custom code
to implement the notification-and-rerender part of this implementation.
Zulip's message editing feature) in order to avoid needing custom code
to implement the notification-and-rerender part of this implementation.
## Soft deactivation
@@ -328,43 +330,43 @@ organization for a few weeks, they are tagged as soft-deactivated.
The way this works internally is:
- We (usually) skip creating UserMessage rows for soft-deactivated
users when a message is sent to a stream where they are subscribed.
users when a message is sent to a stream where they are subscribed.
- If/when the user ever returns to Zulip, we can at that time
reconstruct the UserMessage rows that they missed, and create the rows
at that time (or, to avoid a latency spike if/when the user returns to
Zulip, this work can be done in a nightly cron job). We can construct
those rows later because we already have the data for when the user
might have been subscribed or unsubscribed from streams by other
users, and, importantly, we also know that the user didn’t interact
with the UI since the message was sent (and thus we can safely assume
that the messages have not been marked as read by the user). This is
done in the `add_missing_messages` function, which is the core of the
soft-deactivation implementation.
reconstruct the UserMessage rows that they missed, and create the rows
at that time (or, to avoid a latency spike if/when the user returns to
Zulip, this work can be done in a nightly cron job). We can construct
those rows later because we already have the data for when the user
might have been subscribed or unsubscribed from streams by other
users, and, importantly, we also know that the user didn’t interact
with the UI since the message was sent (and thus we can safely assume
that the messages have not been marked as read by the user). This is
done in the `add_missing_messages` function, which is the core of the
soft-deactivation implementation.
- The “usually” above is because there are a few flags that result
from content in the message (e.g., a message that mentions a user
results in a “mentioned” flag in the UserMessage row), that we need to
keep track of. Since parsing a message can be expensive (>10ms of
work, depending on message content), it would be too inefficient to
need to re-parse every message when a soft-deactivated user comes back
to Zulip. Conveniently, those messages are rare, and so we can just
create UserMessage rows which would have “interesting” flags at the
time they were sent without any material performance impact. And then
`add_missing_messages` skips any messages that already have a
`UserMessage` row for that user when doing its backfill.
from content in the message (e.g., a message that mentions a user
results in a “mentioned” flag in the UserMessage row), that we need to
keep track of. Since parsing a message can be expensive (>10ms of
work, depending on message content), it would be too inefficient to
need to re-parse every message when a soft-deactivated user comes back
to Zulip. Conveniently, those messages are rare, and so we can just
create UserMessage rows which would have “interesting” flags at the
time they were sent without any material performance impact. And then
`add_missing_messages` skips any messages that already have a
`UserMessage` row for that user when doing its backfill.
The end result is the best of both worlds:
- Nobody's view of the world is different because the user was
soft-deactivated (resulting in no visible user-experience impact), at
least if one is running the cron job. If one does not run the cron
job, then users returning after being away for a very long time will
potentially have a (very) slow loading experience as potentially
100,000s of UserMessage rows might need to be reconstructed at once.
soft-deactivated (resulting in no visible user-experience impact), at
least if one is running the cron job. If one does not run the cron
job, then users returning after being away for a very long time will
potentially have a (very) slow loading experience as potentially
100,000s of UserMessage rows might need to be reconstructed at once.
- On the latency-sensitive message sending and fanout code path, the
server only needs to do work for users who are currently interacting
with Zulip.
server only needs to do work for users who are currently interacting
with Zulip.
Empirically, we've found this technique completely resolved the "send
latency" scaling problem. The latency of sending a message to a stream
@@ -374,6 +376,7 @@ it’ll arrive in the couple hundred milliseconds one would expect if
the extra 4500 inactive subscribers didn’t exist.
There are a few details that require special care with this system:
- [Email and mobile push
notifications](../subsystems/notifications.md). We need to make
sure these are still correctly delivered to soft-deactivated users;

7
docs/subsystems/settings.md
View File

@@ -5,6 +5,7 @@ help you decide how to correctly implement new settings you're adding
to Zulip.
We have two types of administrative settings in Zulip:
- **Server settings** are set via configuration files, and apply to
the whole Zulip installation.
- **Realm settings** (or **organization settings**) are usually
@@ -16,11 +17,11 @@ Philosophically, the goals of the settings system are to make it
convenient for:
- Zulip server administrators to configure
Zulip's featureset for their server without needing to patch Zulip
Zulip's featureset for their server without needing to patch Zulip
- Realm administrators to configure settings for their organization
independently without needing to talk with the server administrator.
independently without needing to talk with the server administrator.
- Secrets (passwords, API keys, etc.) to be stored in a separate place
from shareable configuration.
from shareable configuration.
## Server settings

15
docs/subsystems/widgets.md
View File

@@ -249,7 +249,7 @@ inside a `choices` list inside of the JSON payload.
Here is what an example payload looks like:
~~~ json
```json
{
"extra_data": {
"type": "choices",
@@ -283,7 +283,7 @@ Here is what an example payload looks like:
},
"widget_type": "zform"
}
~~~
```
When users click on the buttons, **generic** click
handlers automatically simulate a client reply using
@@ -301,12 +301,11 @@ are completely generic.
We can walk through the steps from the bot generating
the **zform** to the client rendering it.
First,
[here](https://github.com/zulip/python-zulip-api/blob/main/zulip_bots/zulip_bots/bots/trivia_quiz/trivia_quiz.py)
is the code that produces the JSON.
``` py
```py
def format_quiz_for_widget(quiz_id: str, quiz: Dict[str, Any]) -> str:
widget_type = 'zform'
question = quiz['question']
@@ -366,16 +365,16 @@ sibling of **poll** and **zform** just has a somewhat more
generic job to do.) In `static/js/widgetize.js` you will see
where this code converges, with snippets like this:
~~~ js
```js
widgets.poll = poll_widget;
widgets.todo = todo_widget;
widgets.zform = zform;
~~~
```
The code in `static/js/zform.js` renders the form (not
shown here) and then sets up a click handler like below:
~~~ js
```js
elem.find('button').on('click', function (e) {
e.stopPropagation();
@@ -391,7 +390,7 @@ shown here) and then sets up a click handler like below:
content: reply_content,
});
});
~~~
```
And then we are basically done!

32
docs/testing/continuous-integration.md
View File

@@ -14,34 +14,34 @@ and false positives, both of which can waste a lot of developer time.
There are a few implications of this overall goal:
- If a test is failing nondeterministically in CI, we consider that to
be an urgent problem.
be an urgent problem.
- If the tests become a lot slower, that is also an urgent problem.
- Everything we do in CI should also have a way to run it quickly
(under 1 minute, preferably under 3 seconds), in order to iterate fast
in development. Except when working on the CI configuration itself, a
developer should never have to repeatedly wait 10 minutes for a full CI
run to iteratively debug something.
(under 1 minute, preferably under 3 seconds), in order to iterate fast
in development. Except when working on the CI configuration itself, a
developer should never have to repeatedly wait 10 minutes for a full CI
run to iteratively debug something.
## GitHub Actions
### Useful debugging tips and tools
- GitHub Actions stores timestamps for every line in the logs. They
are hidden by default; you can see them by toggling the
`Show timestamps` option in the menu on any job's log page. (You can
get this sort of timestamp in a development environment by piping
output to `ts`).
are hidden by default; you can see them by toggling the
`Show timestamps` option in the menu on any job's log page. (You can
get this sort of timestamp in a development environment by piping
output to `ts`).
- GitHub Actions runs on every branch you push on your Zulip fork.
This is helpful when debugging something complicated.
This is helpful when debugging something complicated.
- You can also ssh into a container to debug failures. SSHing into
the containers can be helpful, especially in rare cases where the
tests are passing in your computer but failing in the CI. There are
various
[Actions](https://github.com/marketplace?type=actions&query=debug+ssh)
available on GitHub Marketplace to help you SSH into a container. Use
whichever you find easiest to set up.
the containers can be helpful, especially in rare cases where the
tests are passing in your computer but failing in the CI. There are
various
[Actions](https://github.com/marketplace?type=actions&query=debug+ssh)
available on GitHub Marketplace to help you SSH into a container. Use
whichever you find easiest to set up.
### Suites

2
docs/testing/linters.md
View File

@@ -46,6 +46,7 @@ You can also run them individually or pass specific files:
`./tools/lint` has many useful options; you can read about them in its
internal documentation using `./tools/lint --help`. Of particular
note are:
- `--fix`: Several of our linters support automatically fixing basic
issues; this option will ask `tools/lint` to run those.
- `--verbose`: Provides detailed information on how to fix many common
@@ -191,6 +192,7 @@ that we exempt may be deemed not worthwhile to fix.
#### JavaScript code
We check our JavaScript code in a few different ways:
- We run eslint.
- We check code formatting with Prettier.
- We perform custom Zulip regex checks on the code.

446
docs/testing/manual-testing.md
View File

@@ -1,4 +1,4 @@
# Manual testing #
# Manual testing
As a general rule, we like to have automated tests for everything that
can be practically tested. However, there are certain types of bugs
@@ -11,7 +11,7 @@ This doc assumes you know how to set up a local development server
and open the Zulip app in the browser. It also assumes a basic
knowledge of how to use Zulip.
## Basic stuff ##
## Basic stuff
When testing Zulip manually, here are things to focus on:
@@ -36,12 +36,13 @@ The rest of this document groups tasks into basic areas of
functionality of the system. If you have multiple people testing
at once, you can divvy up QA tasks by these sections in the doc.
### Message view ###
### Message view
We mostly test the message view as part of testing everything
else, but there are few things to specially test here.
Try using all the navigation hotkeys:
- Up/k
- Down/j
- PgUp/K
@@ -50,103 +51,107 @@ Try using all the navigation hotkeys:
- also try scrolling aggressively with the mouse
Try narrowing from the message view:
- Hotkeys
- use a to go to All messages
- use s to narrow to a stream (select message first
and verify in sidebar)
- use S to narrow to the topic (and verify in sidebar)
- use v to navigate to private messages
- Click on the recipient bar
- narrow to a stream
- narrow to a topic
- narrow to PMs with one user
- narrow to a group PM
- Click on the Zulip logo
- narrow to a topic
- click on the Zulip logo (and verify you're in the Recent topics view)
### Messagebox ###
- Hotkeys
- use a to go to All messages
- use s to narrow to a stream (select message first
and verify in sidebar)
- use S to narrow to the topic (and verify in sidebar)
- use v to navigate to private messages
- Click on the recipient bar
- narrow to a stream
- narrow to a topic
- narrow to PMs with one user
- narrow to a group PM
- Click on the Zulip logo
- narrow to a topic
- click on the Zulip logo (and verify you're in the Recent topics view)
### Messagebox
With messagebox we want to test mainly the functioning of all
all the keyboard shortcuts and click handlers.
Apart from that there are three views of a message box, we want
to test their appearance too:
- Message that includes sender
- Message without the sender
- "/me" message
Here's how we're going to test the message appearances:
- narrow to a new topic and send a message (this message will include sender)
- edit the message ("(EDITED)" label should appear beside sender name)
- edit the message ("(EDITED)" label should appear beside sender name)
- send another message (will not include sender)
- edit the message ("(EDITED)" label should appear in the left column, where the avatar is)
- edit the message ("(EDITED)" label should appear in the left column, where the avatar is)
- send a "/me" message (`/me test message`)
- message should appear alongside sender name
- edit the message ("(EDITED)" label should appear beside the message)
- message should appear alongside sender name
- edit the message ("(EDITED)" label should appear beside the message)
For all the three cases, we need to test the click handlers and
the hotkeys too:
- Sender popover:
- click on the avatar and sender name for messages which include sender
- press 'u' to open the sender popover for all messages
- click on the avatar and sender name for messages which include sender
- press 'u' to open the sender popover for all messages
- Message reply:
- click on message to reply
- use 'r' or Return hotkey to reply
- use '>' to quote and reply
- use '@' to mention and reply
- click on message to reply
- use 'r' or Return hotkey to reply
- use '>' to quote and reply
- use '@' to mention and reply
- Reactions:
- click on the reactions button to open menu
- use ':' to open the reactions menu
- react to a message
- click on the reactions button to open menu
- use ':' to open the reactions menu
- react to a message
- Chevron
- click on chevron to open menu
- use 'i' to open chevron menu
- click on chevron to open menu
- use 'i' to open chevron menu
- Message edit
- click on message edit/view source button
- use 'i' + Return to edit/view source message
- click on the 'copy and close' option in view source and verify positioning of 'Copied!' label.
- click on message edit/view source button
- use 'i' + Return to edit/view source message
- click on the 'copy and close' option in view source and verify positioning of 'Copied!' label.
- Star a message:
- click on the star button in the right column
- use 'Ctrl + S' to star a message
- click on the star button in the right column
- use 'Ctrl + S' to star a message
- Message length
- send a long message and see if '[More]' appears
- click on the 'more' or 'collapse' link
- use i to collapse/expand a message irrespective of message length
- send a long message and see if '[More]' appears
- click on the 'more' or 'collapse' link
- use i to collapse/expand a message irrespective of message length
- use 'v' to show all images in the thread
- use 'M' to mute the thread
Play with the screen size to check if the messagebox appears
fine in different screens too.
### Message editing ###
### Message editing
With message editing we mainly want to exercise topic changes.
Here are some tasks:
- Do lots of editing
- send a message to the topic "original"
- edit the message content
- send two messages to the "original" stream
- start to edit a message but then cancel
- change the topic for the first message to "change1" (just
this message)
- narrow back to "original"
- send one more message to the stream
- change the topic for the last two messages to "change2"
- narrow back to "original"
- send two more messages to the stream
- edit the 2nd message on topic and change all messages to
"change3"
- send a message to the topic "original"
- edit the message content
- send two messages to the "original" stream
- start to edit a message but then cancel
- change the topic for the first message to "change1" (just
this message)
- narrow back to "original"
- send one more message to the stream
- change the topic for the last two messages to "change2"
- narrow back to "original"
- send two more messages to the stream
- edit the 2nd message on topic and change all messages to
"change3"
- Test UI entry points
- hit "i" then down arrow to edit with the popup
- use the popup using the mouse
- enter edit mode using the pencil icon
- hit "i" then down arrow to edit with the popup
- use the popup using the mouse
- enter edit mode using the pencil icon
### Narrowing ###
### Narrowing
Zulip uses the term "narrowing" to refer to opening different views
of your messages, whether by clicking on sidebar options, recipient
@@ -155,7 +160,6 @@ be watching unread counts. Of course, you also want to see messages
show up in the message pane. And, finally, you should make sure
that no messages outside the narrow show up in Cordelia's view.
```eval_rst
.. important::
Make sure that Cordelia is subscribed to Verona but not
@@ -195,7 +199,7 @@ There are 56 things to test here. If you can get into a rhythm
where you can test each case in about 30 seconds, then the whole
exercise is about 30 minutes, assuming no bugs.
### Composing messages ###
### Composing messages
We have pretty good automated tests for our Markdown processor, so
manual testing is targeted more to other interactions. For composing
@@ -203,59 +207,65 @@ a message, pay attention to details like what is automatically
populated and where the focus is placed.
- Hotkeys
- use r to reply to a stream message
- use r to reply to a PM
- use R to reply to the author of a PM
- use R to reply to the author of a PM stream
- use c to compose a stream message
- use x to compose a new PM
- use r to reply to a stream message
- use r to reply to a PM
- use R to reply to the author of a PM
- use R to reply to the author of a PM stream
- use c to compose a stream message
- use x to compose a new PM
- Buttons
- Narrow to a stream and click on "New topic"
- Narrow "Private messages" and click on "New topic"
- Narrow to a stream and click on "New private message"
- Narrow "Private messages" and click on "New private message"
- Narrow to a stream and click on "New topic"
- Narrow "Private messages" and click on "New topic"
- Narrow to a stream and click on "New private message"
- Narrow "Private messages" and click on "New private message"
- Topics
- Compose/send a message to a stream with no topic.
- Compose/send a message to a stream with a new topic.
- Compose/send a message to a stream with autocomplete.
- Compose/send a message to a stream manually typing an
existing topic.
- Compose/send a message to a stream with no topic.
- Compose/send a message to a stream with a new topic.
- Compose/send a message to a stream with autocomplete.
- Compose/send a message to a stream manually typing an
existing topic.
- Formatting stuff
- Use the "A" icon to get Markdown help.
- Use the eyeball icon to show a preview and send from preview mode.
- Toggle in and out of preview before sending a message.
- Use @-mention to mention Hamlet (and send him a message).
- Use `#**devel**` syntax and send to Hamlet, then follow the link.
- Create a bulleted list.
- Use the emoji icon to find an emoji in the picker.
- Use the "A" icon to get Markdown help.
- Use the eyeball icon to show a preview and send from preview mode.
- Toggle in and out of preview before sending a message.
- Use @-mention to mention Hamlet (and send him a message).
- Use `#**devel**` syntax and send to Hamlet, then follow the link.
- Create a bulleted list.
- Use the emoji icon to find an emoji in the picker.
- Attachments
- Send a message with an attachment using the paperclip icon.
- Send a message with multiple attachments.
- Copy an image from the clipboard.
- Use drag/drop from the desktop to upload an image.
- Send a message with an attachment using the paperclip icon.
- Send a message with multiple attachments.
- Copy an image from the clipboard.
- Use drag/drop from the desktop to upload an image.
- Drafts
- Start composing a message then click outside the compose box.
- Use "restore drafts" to restore the draft.
- Start composing then use "Esc" to abort the message.
- Use "restore drafts" to restore the draft.
- Start composing a stream message and then abort using
the little "x" icon in the compose box.
- Click on "New private message" and restore the draft. (You
should now be sending to a stream.)
- Start composing a message then click outside the compose box.
- Use "restore drafts" to restore the draft.
- Start composing then use "Esc" to abort the message.
- Use "restore drafts" to restore the draft.
- Start composing a stream message and then abort using
the little "x" icon in the compose box.
- Click on "New private message" and restore the draft. (You
should now be sending to a stream.)
- Click to send
- Turn off Enter-to-send.
- Send a two-paragraph message using Tab and Enter.
- Send a two-paragraph message using Ctrl-Enter or Cmd-Enter.
- Turn on Enter-to-send.
- Hit Enter to send.
- Turn off Enter-to-send.
- Send a two-paragraph message using Tab and Enter.
- Send a two-paragraph message using Ctrl-Enter or Cmd-Enter.
- Turn on Enter-to-send.
- Hit Enter to send.
### Popover menus ###
### Popover menus
For this task you just want to go through all of our popover menus
and exercise them. The main nuance here is that you occasionally want
@@ -267,66 +277,70 @@ then the message will disappear from the view.
Here are the things to test:
- Stream sidebar menus (click ellipsis when hovering over stream filters)
- Stream settings (just make sure it goes there)
- Narrow (and then have Hamlet send a message)
- Pin/unpin (do both)
- Compose (send a message to the stream)
- Mark as read (scroll back and then have Hamlet send you a message)
- Mute/unmute (do both)
- Unsubscribe (and then go to Stream settings in the gear menu to resubscribe)
- Choose custom color (play around with this)
- Stream settings (just make sure it goes there)
- Narrow (and then have Hamlet send a message)
- Pin/unpin (do both)
- Compose (send a message to the stream)
- Mark as read (scroll back and then have Hamlet send you a message)
- Mute/unmute (do both)
- Unsubscribe (and then go to Stream settings in the gear menu to resubscribe)
- Choose custom color (play around with this)
- Topic sidebar menus (click ellipsis when hovering over topics)
- Narrow (and then have Hamlet send a message)
- Mute/unmute (try both)
- Mark as read (scroll back and then have Hamlet send you a message)
- Narrow (and then have Hamlet send a message)
- Mute/unmute (try both)
- Mark as read (scroll back and then have Hamlet send you a message)
- Left-message-pane menus (click on person's name)
- Verify email
- Verify date message sent
- Send a PM (make sure compose box is filled out ok)
- Narrow to PMs with
- Narrow to PMs sent by
- Verify email
- Verify date message sent
- Send a PM (make sure compose box is filled out ok)
- Narrow to PMs with
- Narrow to PMs sent by
- Right-pane-pane menus (click on chevron when hovering)
- use "i" hotkey to open the menu
- Edit a message you sent (using the down-arrow key to navigate the popup)
- View Source for somebody else's message (make sure
it's not editable)
- Reply (send a message)
- Collapse/uncollapse (try both)
- Mute/unmute (try both, watch left sidebar)
- Link to this conversation
- use "i" hotkey to open the menu
- Edit a message you sent (using the down-arrow key to navigate the popup)
- View Source for somebody else's message (make sure
it's not editable)
- Reply (send a message)
- Collapse/uncollapse (try both)
- Mute/unmute (try both, watch left sidebar)
- Link to this conversation
- Buddy list menus (click ellipsis when hovering over users)
- Narrow to PMs with
- Narrow to message sent by
- Compose a message to
- Narrow to PMs with
- Narrow to message sent by
- Compose a message to
### Sidebar filtering ###
### Sidebar filtering
This is a fairly quick task where we test the search filters on the left sidebar
and the buddy list. If Cordelia is not subscribed to Denmark, subscribe her to
that stream.
- Streams filtering
- Use "w" hotkey to open the search.
- Filter on "d".
- Pin/unpin Denmark.
- Clear filter.
- Use "A" and "D" hotkeys to cycle through the streams.
- Filter again and then click somewhere else.
- Use "w" hotkey to open the search.
- Filter on "d".
- Pin/unpin Denmark.
- Clear filter.
- Use "A" and "D" hotkeys to cycle through the streams.
- Filter again and then click somewhere else.
- Buddy list filtering
- Use "q" hotkey to open the search.
- Filter for Hamlet, Prospero, Othello, etc.
- Log on Hamlet and log off Hamlet while filtering for Hamlet.
- Log on/log off Hamlet while filtering for Othello.
- Log on/log off Hamlet while not filtering at all.
- Filter again and then click somewhere else.
- Use "q" hotkey to open the search.
- Filter for Hamlet, Prospero, Othello, etc.
- Log on Hamlet and log off Hamlet while filtering for Hamlet.
- Log on/log off Hamlet while filtering for Othello.
- Log on/log off Hamlet while not filtering at all.
- Filter again and then click somewhere else.
### Stream permissions ###
### Stream permissions
This is an important category to test, because we obviously do not
want to have bugs where people can read messages on streams they
@@ -338,16 +352,16 @@ that Cordelia has the correct visibility to them.
First, we start off with "positive" tests.
- Positive tests
- Have Hamlet create a public stream w/Cordelia subscribed and
have him post a message to the stream.
- Have Hamlet create a public stream without Cordelia and then...
- Have Hamlet post to the stream.
- Have Cordelia subscribe to the stream.
- Verify Cordelia can see the previous message.
- Have Cordelia post a message to the stream.
- Have Hamlet create a private stream with Cordelia
invited and test a two-way conversation between the two
users.
- Have Hamlet create a public stream w/Cordelia subscribed and
have him post a message to the stream.
- Have Hamlet create a public stream without Cordelia and then...
- Have Hamlet post to the stream.
- Have Cordelia subscribe to the stream.
- Verify Cordelia can see the previous message.
- Have Cordelia post a message to the stream.
- Have Hamlet create a private stream with Cordelia
invited and test a two-way conversation between the two
users.
For negative tests, we want to dig a little deeper to find back
doors for Cordelia to access the stream. Here are some techniques
@@ -364,21 +378,21 @@ and she can subsequently subscribe. For private streams, she should
not even know they exist (until she's invited, of course).
- Negative tests
- Have Hamlet create a public stream without inviting Cordelia.
- Verify Cordelia can see the stream in her settings.
- Verify Cordelia can't compose a message to the stream.
- Verify that Cordelia sees nothing when Hamlet posts to
the stream.
- Have Hamlet create a public stream with Cordelia, but then
have Iago revoke her subscription using the admin page.
- Verify that the stream appears in Cordelia's left sidebar
and then goes away.
- Try to have Cordelia view the stream using a sneaky
search along the lines of `stream:foo`.
- Have Hamlet create a private stream without inviting Cordelia.
- Verify Cordelia can't compose a message to the stream.
- Have Hamlet create a public stream without inviting Cordelia.
- Verify Cordelia can see the stream in her settings.
- Verify Cordelia can't compose a message to the stream.
- Verify that Cordelia sees nothing when Hamlet posts to
the stream.
- Have Hamlet create a public stream with Cordelia, but then
have Iago revoke her subscription using the admin page.
- Verify that the stream appears in Cordelia's left sidebar
and then goes away.
- Try to have Cordelia view the stream using a sneaky
search along the lines of `stream:foo`.
- Have Hamlet create a private stream without inviting Cordelia.
- Verify Cordelia can't compose a message to the stream.
### Search ###
### Search
The main task for testing search is to play around with search
suggestions (autocomplete). Once you select an option, verify the
@@ -387,6 +401,7 @@ reflects the current narrow. If a search comes up legitimately
empty, have Hamlet send a message that matches the search.
Here are searches you should be able to do with autocomplete:
- stream:design
- stream:Verona topic:Verona1
- stream:Verona keyword
@@ -398,17 +413,20 @@ Here are searches you should be able to do with autocomplete:
- PMs with Hamlet matching keyword "foo"
There are some things you can try that don't come up in autocomplete:
- -stream:Verona (exclude Verona)
- stream:Verona stream:devel (should return no results)
Miscellaneous:
- Use the "/" hotkey to start a search.
- Use the "x" icon to clear a search.
- Use the "Esc" hotkey to clear a search.
### Stream settings ###
### Stream settings
Test various UI entry points into stream settings:
- Use small gear menu in left sidebar, then filter to "devel".
- Use popover menu in left sidebar next to "devel".
- Use gear menu above buddy list and filter to "devel".
@@ -417,6 +435,7 @@ Test various UI entry points into stream settings:
(I'm not sure why we still have the chevron at this writing.)
Create new public stream "public1" and add Hamlet:
- Type "public1" in the text box and then click "Create new stream."
- Select "People must be invited" and then verify you can't
select "Announce stream".
@@ -425,6 +444,7 @@ Create new public stream "public1" and add Hamlet:
- Hit the "Create" button.
Test subscribe/unsubscribe:
- Log in as Hamlet and go to his stream settings.
- As Cordelia, unsubscribe from "public1" using the checkmark in the
streams settings page.
@@ -435,16 +455,18 @@ Test subscribe/unsubscribe:
As Cordelia, exercise different options in Create Stream
dialog by creating streams s1, s2, s3, etc.:
- s1: anyone can join, announce it, and add Hamlet using filter feature
- s2: people must be invited
- s3: anyone can join, don't announce
- s4: check all, then uncheck all, then invite only Hamlet
- s5: invite everybody but Hamlet
- s6:
- create the stream as public, but don't subscribe anybody initially
- then click on stream options to add Hamlet using "Add" button
- create the stream as public, but don't subscribe anybody initially
- then click on stream options to add Hamlet using "Add" button
Test per-stream options:
- Use "devel" stream and send a message to it
- Do mute and unmute, have Hamlet send messages
- Test notifications on/off, have Hamlet send messages
@@ -453,50 +475,51 @@ Test per-stream options:
messages view
- Verify stream subscriber counts in the main stream view
### User settings ###
### User settings
You can modify per-user settings by choosing "Settings" in the gear menu.
Do these tasks as Cordelia.
- Your account
- Change full name (Hamlet should see the name change)
- Customize profile picture
- Deactivate account (and then log in as Iago to re-activate Cordelia)
- Change full name (Hamlet should see the name change)
- Customize profile picture
- Deactivate account (and then log in as Iago to re-activate Cordelia)
- Display settings
- Right now, these unfortunately require reloads to take effect.
- Default language (change to Spanish)
- Show user list on left sidebar in narrow windows (verify by making window thinner)
- 24-hour time (and then test going back to AM/PM)
- Right now, these unfortunately require reloads to take effect.
- Default language (change to Spanish)
- Show user list on left sidebar in narrow windows (verify by making window thinner)
- 24-hour time (and then test going back to AM/PM)
- Notifications
- Stream message
- turn off notifications at user level
- create a new stream
- have Hamlet send a message
- turn on notifications at user level
- create a new stream
- have Hamlet send a message
- then turn off notifications for that stream
- have Hamlet send another message
- Private messages and @-mentions
- Test Desktop/Audible options
- You can ignore other stuff for now
- Stream message
- turn off notifications at user level
- create a new stream
- have Hamlet send a message
- turn on notifications at user level
- create a new stream
- have Hamlet send a message
- then turn off notifications for that stream
- have Hamlet send another message
- Private messages and @-mentions
- Test Desktop/Audible options
- You can ignore other stuff for now
- Bots/API key
- Create a bot with a generic avatar and send it a PM
- Create a bot with a custom avatar and send it a PM
- Change your API key
- Create a bot with a generic avatar and send it a PM
- Create a bot with a custom avatar and send it a PM
- Change your API key
- Alert words
- Create an alert word
- Have Hamlet send you a message that includes the alert word
- Create an alert word
- Have Hamlet send you a message that includes the alert word
- Zulip labs
- Turn on auto-scroll to new messages (and have Hamlet send you one)
- Turn on/off "Enable desktop notifications for new streams" and test.
(We may eliminate this option soon.)
- Turn on auto-scroll to new messages (and have Hamlet send you one)
- Turn on/off "Enable desktop notifications for new streams" and test.
(We may eliminate this option soon.)
### Keyboard shortcuts ###
### Keyboard shortcuts
We mostly test keyboard shortcuts as part of other tasks.
Here are the tasks for this section:
- Use the "?" hotkey to open the keyboard help
- Proofread the dialog for typos.
- Close the dialog.
@@ -504,20 +527,22 @@ Here are the tasks for this section:
- Find a hotkey that you don't frequently use and experiment with its
usage.
### Miscellaneous menu options ###
### Miscellaneous menu options
Make sure that these options launch appropriate help screens:
- Proofread and try a couple random options:
- Message formatting
- Search operators
- Make sure help launches in a separate browser tab:
- Desktop and mobile apps
- Integrations
- API documentation
### Inviting users/tutorial ###
- Proofread and try a couple random options:
- Message formatting
- Search operators
- Make sure help launches in a separate browser tab:
- Desktop and mobile apps
- Integrations
- API documentation
### Inviting users/tutorial
Here are the tasks:
- Invite ignore@zulip.com using the link beneath the buddy list but
then don't take further action.
- Fully invite foo@zulip.com using the gear menu.
@@ -529,10 +554,11 @@ Here are the tasks:
much of our production code, since the login flow is customized for
the development environment).
### To be continued... ###
### To be continued...
This document does not cover settings/admin options yet. The main
things to do when testing the settings system are:
- Verify that changes are synced to other users.
- Verify error messages appear if you do something wrong and look right.
- For organization settings, verify that they look right in read-only

15
docs/testing/philosophy.md
View File

@@ -61,18 +61,19 @@ suite (`test-js-with-node`) to run in under 10 seconds.
It'd be a long blog post to summarize everything we do to help achieve
these goals, but a few techniques are worth highlighting:
- Our test suites are designed to not access the Internet, since the
Internet might be down or unreliable in the test environment. Where
outgoing HTTP requests are required to test something, we mock the
responses with libraries like `responses`.
- We carefully avoid the potential for contamination of data inside
services like PostgreSQL, Redis, and memcached from different tests.
- Every test case prepends a unique random prefix to all keys it
uses when accessing Redis and memcached.
- Every test case runs inside a database transaction, which is
aborted after the test completes. Each test process interacts
only with a fresh copy of a special template database used for
server tests that is destroyed after the process completes.
- Every test case prepends a unique random prefix to all keys it
uses when accessing Redis and memcached.
- Every test case runs inside a database transaction, which is
aborted after the test completes. Each test process interacts
only with a fresh copy of a special template database used for
server tests that is destroyed after the process completes.
- We rigorously investigate non-deterministically failing tests as though
they were priority bugs in the product.
@@ -141,6 +142,7 @@ Some examples of this philosophy:
run, and produce this output".
In the Zulip context:
- Zulip uses the same API for our webapp as for our mobile clients and
third-party API clients, and most of our server tests are written
against the Zulip API.
@@ -150,6 +152,7 @@ In the Zulip context:
message as output, to test the actual interface.
So, to summarize our approach to integration vs. unit testing:
- While we aim to achieve test coverage of every significant code path
in the Zulip server, which is commonly associated with unit testing,
most of our tests are integration tests in the sense of sending a

59
docs/testing/testing-with-django.md
View File

@@ -120,11 +120,12 @@ Here are some example action methods that tests may use for data setup:
Some tests need to access the filesystem (e.g. `test_upload.py` tests
for `LocalUploadBackend` and the data import tests). Doing
this correctly requires care to avoid problems like:
- Leaking files after every test (which are clutter and can eventually
run the development environment out of disk) or
run the development environment out of disk) or
- Interacting with other parallel processes of this `test-backend` run
(or another `test-backend` run), or with later tests run by this
process.
(or another `test-backend` run), or with later tests run by this
process.
To avoid these problems, you can do the following:
@@ -151,8 +152,8 @@ techniques.
#### What is mocking?
When writing tests, *mocks allow you to replace methods or objects with fake entities
suiting your testing requirements*. Once an object is mocked, **its original code does not
When writing tests, _mocks allow you to replace methods or objects with fake entities
suiting your testing requirements_. Once an object is mocked, **its original code does not
get executed anymore**.
Rather, you can think of a mocked object as an initially empty shell:
@@ -201,12 +202,12 @@ def greet(name_key: str) -> str:
```
-> **You have a problem**: `greet()` calls `fetch_database()`. `fetch_database()` does some look-ups in
a database. *You haven't created that database for your tests, so your test would fail, even though
the code is correct.*
a database. _You haven't created that database for your tests, so your test would fail, even though
the code is correct._
- Luckily, you know that `fetch_database("Mario")` should return "Mr. Mario Mario".
- *Hint*: Sometimes, you might not know the exact return value, but one that is equally valid and works
- _Hint_: Sometimes, you might not know the exact return value, but one that is equally valid and works
with the rest of the code. In that case, just use this one.
-> **Solution**: You mock `fetch_database()`. This is also referred to as "mocking out" `fetch_database()`.
@@ -233,7 +234,7 @@ It also implements `MagicMock`, which is the same as `Mock`, but contains many d
those are the ones starting with with a dunder `__`). From the docs:
> In most of these examples the Mock and MagicMock classes are interchangeable. As the MagicMock is the more capable class
it makes a sensible one to use by default.
> it makes a sensible one to use by default.
`Mock` itself is a class that principally accepts and records any and all calls. A piece of code like
@@ -246,7 +247,7 @@ foo.baz
foo.qux = 42
```
is *not* going to throw any errors. Our mock silently accepts all these calls and records them.
is _not_ going to throw any errors. Our mock silently accepts all these calls and records them.
`Mock` also implements methods for us to access and assert its records, e.g.
```python
@@ -255,8 +256,8 @@ foo.bar.assert_called_with('quux')
Finally, `unittest.mock` also provides a method to mock objects only within a scope: `patch()`. We can use `patch()` either
as a decorator or as a context manager. In both cases, the mock created by `patch()` will apply for the scope of the decorator /
context manager. `patch()` takes only one required argument `target`. `target` is a string in dot notation that *refers to
the name of the object you want to mock*. It will then assign a `MagicMock()` to that object.
context manager. `patch()` takes only one required argument `target`. `target` is a string in dot notation that _refers to
the name of the object you want to mock_. It will then assign a `MagicMock()` to that object.
As an example, look at the following code:
```python
@@ -269,7 +270,7 @@ with mock.patch('__main__.urandom', return_value=42):
print(urandom(1)) # We exited the context manager, so the mock doesn't apply anymore. Will return a random byte.
```
*Note that calling `mock.patch('os.urandom', return_value=42)` wouldn't work here*: `os.urandom` would be the name of our patched
_Note that calling `mock.patch('os.urandom', return_value=42)` wouldn't work here_: `os.urandom` would be the name of our patched
object. However, we imported `urandom` with `from os import urandom`; hence, we bound the `urandom` name to our current module
`__main__`.
@@ -454,28 +455,28 @@ to verify that the endpoint is properly failing.
Here are some things to consider when writing new tests:
- **Duplication** We try to avoid excessive duplication in tests.
If you have several tests repeating the same type of test setup,
consider making a setUp() method or a test helper.
If you have several tests repeating the same type of test setup,
consider making a setUp() method or a test helper.
- **Network independence** Our tests should still work if you don't
have an internet connection. For third party clients, you can simulate
their behavior using fixture data. For third party servers, you can
typically simulate their behavior using mocks.
have an internet connection. For third party clients, you can simulate
their behavior using fixture data. For third party servers, you can
typically simulate their behavior using mocks.
- **Coverage** We have 100% line coverage on several of our backend
modules. You can use the `--coverage` option to generate coverage
reports, and new code should have 100% coverage, which generally
requires testing not only the "happy path" but also error handling
code and edge cases. It will generate a nice HTML report that you can
view right from your browser (the tool prints the URL where the report
is exposed in your development environment).
modules. You can use the `--coverage` option to generate coverage
reports, and new code should have 100% coverage, which generally
requires testing not only the "happy path" but also error handling
code and edge cases. It will generate a nice HTML report that you can
view right from your browser (the tool prints the URL where the report
is exposed in your development environment).
- **Console output** A properly written test should print nothing to
the console; use `with self.assertLogs` to capture and verify any
logging output. Note that we reconfigure various loggers in
`zproject/test_extra_settings.py` where the output is unlikely to be
interesting when running our test suite.
`test-backend --ban-console-output` checks for stray print statements.
the console; use `with self.assertLogs` to capture and verify any
logging output. Note that we reconfigure various loggers in
`zproject/test_extra_settings.py` where the output is unlikely to be
interesting when running our test suite.
`test-backend --ban-console-output` checks for stray print statements.
Note that `test-backend --coverage` will assert that
various specific files in the project have 100% test coverage and

50
docs/testing/testing-with-node.md
View File

@@ -7,6 +7,7 @@ system since it is much (>100x) faster and also easier to do correctly
than the Puppeteer system.
You can run this test suite as follows:
```bash
tools/test-js-with-node
```
@@ -101,10 +102,10 @@ code you're trying to test. For that reason, each unit test file
explicitly declares all of the modules it depends on, with a few
different types of declarations depending on whether we want to:
- Exercise the module's real code for deeper, more realistic testing?
- Stub out the module's interface for more control, speed, and
isolation?
- Do some combination of the above?
- Exercise the module's real code for deeper, more realistic testing?
- Stub out the module's interface for more control, speed, and
isolation?
- Do some combination of the above?
For all the modules where you want to run actual code, add statements
like the following toward the top of your test file:
@@ -164,7 +165,7 @@ tools/test-js-with-node --coverage
If tests pass, you will get instructions to view coverage reports
in your browser.
Note that modules that we don't test *at all* aren't listed in the
Note that modules that we don't test _at all_ aren't listed in the
report, so this tends to overstate how good our overall coverage is,
but it's accurate for individual files. You can also click a filename
to see the specific statements and branches not tested. 100% branch
@@ -191,31 +192,32 @@ These instructions assume you're using the Vagrant development environment.
2. In WebStorm, navigate to `Preferences -> Tools -> Vagrant` and
configure the following:
- `Instance folder` should be the root of the `zulip` repository on
your host (where the Vagrantfile is located).
- `Provider` should be `virtualbox` on macOS and Docker on Linux
- In `Boxes`, choose the one used for Zulip (unless you use
Virtualbox for other things, there should only be one option).
- `Instance folder` should be the root of the `zulip` repository on
your host (where the Vagrantfile is located).
- `Provider` should be `virtualbox` on macOS and Docker on Linux
- In `Boxes`, choose the one used for Zulip (unless you use
Virtualbox for other things, there should only be one option).
You shouldn't need to set these additional settings:
- `Vagrant executable` should already be correctly `vagrant`.
- `Environment Variables` is not needed.
You shouldn't need to set these additional settings:
- `Vagrant executable` should already be correctly `vagrant`.
- `Environment Variables` is not needed.
3. You'll now need to set up a WebStorm "Debug Configuration". Open
the `Run/Debug Configuration` menu and create a new `Node.js` config:
1. Under `Node interpreter:` click the 3 dots to the right side and
1. Under `Node interpreter:` click the 3 dots to the right side and
click on the little plus in the bottom left of the
`Node.js Interpreters` window.
1. Select `Add Remote...`.
1. In the `Configure Node.js Remote Interpreter`, window select `Vagrant`
1. Wait for WebStorm to connect to Vagrant. This will be displayed
by the `Vagrant Host URL` section updating to contain the Vagrant
SSH URL, e.g. `ssh://vagrant@127.0.0.1:2222`.
1. **Set the `Node.js interpreter path` to `/usr/local/bin/node`**
1. Hit `OK` 2 times to get back to the `Run/Debug Configurations` window.
1. Under `Working Directory` select the root `zulip` directory.
1. Under `JavaScript file`, enter `frontend_tests/zjsunit/index.js`
-- this is the root script for Zulip's node unit tests.
1. Select `Add Remote...`.
1. In the `Configure Node.js Remote Interpreter`, window select `Vagrant`
1. Wait for WebStorm to connect to Vagrant. This will be displayed
by the `Vagrant Host URL` section updating to contain the Vagrant
SSH URL, e.g. `ssh://vagrant@127.0.0.1:2222`.
1. **Set the `Node.js interpreter path` to `/usr/local/bin/node`**
1. Hit `OK` 2 times to get back to the `Run/Debug Configurations` window.
1. Under `Working Directory` select the root `zulip` directory.
1. Under `JavaScript file`, enter `frontend_tests/zjsunit/index.js`
-- this is the root script for Zulip's node unit tests.
Congratulations! You've now set up the integration.

1
docs/testing/testing-with-puppeteer.md
View File

@@ -10,6 +10,7 @@ keyboard shortcuts, etc.).
## Running tests
You can run this test suite as follows:
```bash
tools/test-js-with-puppeteer
```

12
docs/testing/testing.md
View File

@@ -140,12 +140,12 @@ depending on Internet access.
This enforcement code results in the following exception:
```pytb
File "tools/test-backend", line 120, in internet_guard
raise Exception("Outgoing network requests are not allowed in the Zulip tests."
Exception: Outgoing network requests are not allowed in the Zulip tests.
...
```
```pytb
File "tools/test-backend", line 120, in internet_guard
raise Exception("Outgoing network requests are not allowed in the Zulip tests."
Exception: Outgoing network requests are not allowed in the Zulip tests.
...
```
#### Documentation tests

3
docs/testing/typescript.md
View File

@@ -15,7 +15,7 @@ discussion and very much subject to change.
A typical piece of TypeScript code looks like this:
``` ts
```ts
setdefault(key: K, value: V): V {
const mapping = this._items[this._munge(key)];
if (mapping === undefined) {
@@ -29,7 +29,6 @@ The following resources are valuable for learning TypeScript:
- The main documentation on [TypeScript syntax][typescript-handbook].
## Type checking
TypeScript types are checked by the TypeScript compiler, `tsc`, which

28
docs/translating/chinese.md
View File

@@ -8,9 +8,9 @@ Zulip is a modern internet application, many Chinese translations are
borrowed from the popular Web software, such as WeiBo, WeChat, QQ
Mail etc. that most Chinese users are familiar with.
Zulip的文风比较口语化，考虑到大多数中国用户的习惯，翻译时的语言习惯稍
微正式了一点，但也尽量避免刻板。Zulip是一款时尚的互联网应用，翻译时也
借鉴了中国用户熟悉的微博、微信、QQ邮箱等软件的用语习惯，以期贴近用户。
Zulip 的文风比较口语化，考虑到大多数中国用户的习惯，翻译时的语言习惯稍
微正式了一点，但也尽量避免刻板。Zulip 是一款时尚的互联网应用，翻译时也
借鉴了中国用户熟悉的微博、微信、QQ 邮箱等软件的用语习惯，以期贴近用户。
## Terms(术语)
@@ -23,9 +23,9 @@ translated as "私信". The domestic WeiBo, WeChat also keep in line
with the habit. "Starred Message" is similar to "Star Mail (星标邮件)"
feature in QQ Mail, so it is translated into "星标消息".
Message可直译为“消息”、“信息”等，两者皆可，这里统一选用“消息”。例如，
Message 可直译为“消息”、“信息”等，两者皆可，这里统一选用“消息”。例如，
“Stream Message”译作“频道消息”；但“Private Message”又译为“私信"，与国
内微博、微信的使用习惯保持一致。“Starred Message”类似于QQ邮箱中的“星标
内微博、微信的使用习惯保持一致。“Starred Message”类似于 QQ 邮箱中的“星标
邮件”功能，这里也借鉴翻译为“星标消息”。
- Stream - **频道**
@@ -41,10 +41,10 @@ group. However, "讨论组" has one more Chinese character than "频道
(Channel)".
曾经使用的翻译有“群组”、“主题”、“版块”，还有“栏目”。现在选择的“频道”灵
感来源于Ingress游戏中的聊天“Channel”。因为“Stream”可以“新建/删除
感来源于 Ingress 游戏中的聊天“Channel”。因为“Stream”可以“新建/删除
（Create/Delete）”、也可以“订阅/退订（Subscribe/Unsubscribe）”，
“Stream”内部还可以发起“话题（Topic）讨论。“Stream”还有一个备选方案，就
是“讨论组”，字多一个，稍微有点啰嗦。主要参考自以前QQ的“讨论组”，在QQ中
是“讨论组”，字多一个，稍微有点啰嗦。主要参考自以前 QQ 的“讨论组”，在 QQ 中
是一种临时的群组。
- Topic - **话题**
@@ -69,7 +69,7 @@ integrating Zulip production with other applications and services. For
integrity in Chinese expression, it is translated as "应用整合
(Application Integration)".
“Integration”原意为“集成”与“整合”，这里表示将其它的应用或者服务与Zulip
“Integration”原意为“集成”与“整合”，这里表示将其它的应用或者服务与 Zulip
实现整合。为表达意思完整，补充翻译为“应用整合”。
- Notification - **通知**
@@ -108,8 +108,8 @@ readability considerations.
1. 在汉语中，“筛选”表示按照指定条件进行挑选的方式。“Narrow to ...”的含
义为“使...缩小范围”，两者有一定共通性。
2. “筛选”也是比较大众化的计算机用语，易于为大家所接受。例如Microsoft
Excel中的“筛选”功能。
2. “筛选”也是比较大众化的计算机用语，易于为大家所接受。例如 Microsoft
Excel 中的“筛选”功能。
另外，在搜索功能的语境中，“Narrow to ...”没有翻译为“筛选”，而翻译为“搜
索”，这是出于可读性的考虑。
@@ -121,7 +121,7 @@ set. Such a translation is not appropriate for Zulip. "开启/关闭免打
扰(Turn off/on Notification)" is a sense to sense translation, which
is also borrowed from the WeChat.
“Mute”常见的中文翻译为“静音”，在电视设备中常见，用在Zulip中并不太合适。
“Mute”常见的中文翻译为“静音”，在电视设备中常见，用在 Zulip 中并不太合适。
这里取意译，与大家常用的微信（WeChat）中“消息免打扰”用语习惯一致。
- Deactivate/Reactivate - **禁用/启用(帐户)，关闭/激活(社区)**
@@ -141,11 +141,11 @@ example "Your realm has been deactivated." to "您的社区已关闭".
for general users. Other translations "错误(Error)", "非法(Illegal)",
"不合法(Invalid)" are all ok. Generally, it is translated as "不正确
(Incorrect)" for consistency. For example, "Invalid API key" is
translated as "API码不正确".
translated as "API 码不正确".
“Invalid”大部分用于一些异常信息，这些信息普通用户应该很少见到。可选翻
译有“错误”、“非法”、“不合法”；为保持一致的习惯，这里统一翻译为“不正确”。
例如“Invalid API key”翻译为“API码不正确”。
例如“Invalid API key”翻译为“API 码不正确”。
- I want - **开启**
@@ -196,6 +196,6 @@ the dot in Chinese (。) often has a bad effect on page layout. It is
recommended to omit the dot, just leave empty at the end of the
sentence or paragraph.
感叹号在Zulip中出现非常多，可能英文中感叹号的语气比中文中略轻一点。在
感叹号在 Zulip 中出现非常多，可能英文中感叹号的语气比中文中略轻一点。在
中文翻译建议省略大部分的感叹号。另外，句号在中文排版中比较影响美观，因
此也一般建议省略不翻。句末留空即可。

4
docs/translating/french.md
View File

@@ -2,10 +2,10 @@
## Rules
- Use of *vous* instead of *tu*,
- Use of _vous_ instead of _tu_,
- A space before and after a colon and a semi-colon and a space after a dot and a comma,
- Follow english capitalization,
- Prefer the infinitive form of a verb: *Save* into *Sauver* (instead of *Sauvez*).
- Prefer the infinitive form of a verb: _Save_ into _Sauver_ (instead of _Sauvez_).
Some translations can be tricky, so please don't hesitate to ask the community or contribute to this guide.

140
docs/translating/german.md
View File

@@ -4,7 +4,6 @@ Thank you for considering to contribute to the German translation!
Before you start writing, please make sure that you have read the
following general translation rules.
## Rules
### Formal or informal?
@@ -17,27 +16,27 @@ in a more colloquial style, German translations should be rather informal as wel
**Don't use slang or regional phrases in the German translation:**
- Instead of *"So'n Dreck kann jedem mal passieren."*, you could
say *"Dieser Fehler tritt häufiger auf."*
- Instead of _"So'n Dreck kann jedem mal passieren."_, you could
say _"Dieser Fehler tritt häufiger auf."_
- "Das ist die Seite, wo der Quelltext steht." - the "*wo*" is regional,
say *"Das ist die Seite, auf der der Quelltext steht."* instead.
- "Das ist die Seite, wo der Quelltext steht." - the "_wo_" is regional,
say _"Das ist die Seite, auf der der Quelltext steht."_ instead.
### Form of address
**Use "Du" instead of "Sie".**
For the reasons provided in [the previous section](#formal-or-informal),
stick to *Du* (informal) instead of *Sie* (formal) when addressing
the reader and remember to capitalize *Du*.
stick to _Du_ (informal) instead of _Sie_ (formal) when addressing
the reader and remember to capitalize _Du_.
### Form of instruction
**Prefer imperative over constructions with auxiliary verbs.**
For instructions, try to use the imperative (e.g. *"Gehe auf die Seite"* -
*"Go to the page"*) instead of constructions with auxiliary verbs
(e.g. *"Du musst auf die Seite ... gehen"* - *"You have to go the page ..."*).
For instructions, try to use the imperative (e.g. _"Gehe auf die Seite"_ -
_"Go to the page"_) instead of constructions with auxiliary verbs
(e.g. _"Du musst auf die Seite ... gehen"_ - _"You have to go the page ..."_).
This keeps the phrases short, less stiff and avoids unnecessary addressing
of the reader.
@@ -47,25 +46,25 @@ of the reader.
To be consistent with other online platforms, use continuous labels for buttons,
item titles, etc. with verbs in infinitive form,
e.g. *Manage streams* - *Kanäle verwalten* instead of *Verwalte Kanäle*.
e.g. _Manage streams_ - _Kanäle verwalten_ instead of _Verwalte Kanäle_.
### Concatenation of words
**Try to avoid it.**
German is famous for its concatenations of nouns
(e.g. *Heizölrückstoßdämpfung*, which means *fuel oil recoil attenuation*).
(e.g. _Heizölrückstoßdämpfung_, which means _fuel oil recoil attenuation_).
For the sake of correct rendering and simplicity, you should try to avoid such
concatenations whenever possible, since they can break the layout of the Zulip
frontend. Try to stick to a maximum length of 20 characters and follow your
intuition.
- A term like *Tastaturkürzel* for *Keyboard shortcuts* is fine - it is
shorter than 20 characters and commonly used in web applications.
- A term like _Tastaturkürzel_ for _Keyboard shortcuts_ is fine - it is
shorter than 20 characters and commonly used in web applications.
- A term like *Benachrichtigungsstichwörter* for *Alert words* should
not be used, it sounds odd and is longer than 20 characters.
You could use "*Stichwörter, die mich benachrichtigen*" instead.
- A term like _Benachrichtigungsstichwörter_ for _Alert words_ should
not be used, it sounds odd and is longer than 20 characters.
You could use "_Stichwörter, die mich benachrichtigen_" instead.
### Anglicisms
@@ -76,21 +75,21 @@ This becomes even more evident in internet applications,
so you should not be afraid of using them if they provide an advantage over
the German equivalent. Take the following two examples as a reference:
- Translating *Stream*: Use the German word *Kanal*, since it is just as short
and used in other web apps.
- Translating _Stream_: Use the German word _Kanal_, since it is just as short
and used in other web apps.
- Translating *Bot*: Use *Bot*, as a completely accurate German
equivalent **doesn't** exist (e.g. *Roboter*) and the term *Bot* is not
unknown to German speakers.
- Translating _Bot_: Use _Bot_, as a completely accurate German
equivalent **doesn't** exist (e.g. _Roboter_) and the term _Bot_ is not
unknown to German speakers.
### Special characters
**Use "ä, ö, ü" and "ß" consistently.**
While *ä, ö, ü* and *ß* are more and more being replaced by *ae, oe, ue*
and *ss* in chats, forums and even websites, German translations
While _ä, ö, ü_ and _ß_ are more and more being replaced by _ae, oe, ue_
and _ss_ in chats, forums and even websites, German translations
containing umlauts have a more trustworthy appearance.
For capitalizations, you can replace the *ß* by *ss*.
For capitalizations, you can replace the _ß_ by _ss_.
### False friends
@@ -100,36 +99,35 @@ A false friend is a word in another language that is spelled
or sounds similar to a word in one's own language,
yet has a different meaning.
False friends for the translation from German to English include
*actually* - *eigentlich*, *eventually* - *schließlich*, *map* - *Karte*, etc.
_actually_ - _eigentlich_, _eventually_ - _schließlich_, _map_ - _Karte_, etc.
Make sure to not walk into such a trap.
### Other
- Try to keep words and phrases short and understandable. The front-end
developers will thank you ;)
developers will thank you ;)
- Be consistent. Use the same terms for the same things, even if that
means repeating. Have a look at other German translations on Zulip
to get a feeling for the vocabulary.
means repeating. Have a look at other German translations on Zulip
to get a feeling for the vocabulary.
- Balance common verbs and nouns with specific IT-related translations
of English terms - this can be tricky, try to check how other resources
were translated (e.g. Gmail, Microsoft websites, Facebook) to decide
what wouldn't sound awkward / rude in German.
of English terms - this can be tricky, try to check how other resources
were translated (e.g. Gmail, Microsoft websites, Facebook) to decide
what wouldn't sound awkward / rude in German.
- For additional translation information, feel free to check out
[this](https://en.wikipedia.org/wiki/Wikipedia:Translating_German_WP) Wikipedia guide
on translating German Wikipedia articles into English.
[this](https://en.wikipedia.org/wiki/Wikipedia:Translating_German_WP) Wikipedia guide
on translating German Wikipedia articles into English.
Some terms are very tricky to translate, so be sure to communicate with other German
speakers in the community. It's all about making Zulip friendly and usable.
## Terms (Begriffe)
- Message - **Nachricht**
*"Nachricht" (Facebook, WhatsApp, Transifex)*
_"Nachricht" (Facebook, WhatsApp, Transifex)_
- Private Message (PM) - **Private Nachricht (PN)**
@@ -137,17 +135,17 @@ Since we try to avoid concatenating words whenever possible, don't use
"Privatnachricht" . PN is the officially used abbreviation for
"Private Nachricht" and is used in many German chat forums.
*"Private Nachricht" (YouTube, Transifex)*
_"Private Nachricht" (YouTube, Transifex)_
- Starred Message - **Markierte Nachricht**
We go with "markiert" instead of "gesternt" (which is not even a proper
German word) here, since it comes closer to the original meaning of "starred".
*"Markierte Nachricht" (Gmail, Transifex),
"Nachricht mit Stern" (WhatsApp)*
_"Markierte Nachricht" (Gmail, Transifex),
"Nachricht mit Stern" (WhatsApp)_
*"Bereich" (Transifex), "Community" (Google+)*
_"Bereich" (Transifex), "Community" (Google+)_
- Stream - **Stream**
@@ -158,24 +156,23 @@ skills, and the best choice for describing Zulip's chat hierarchy. The term
by other chat applications with a simple, flat chat hierarchy, that is,
no differentiation between streams and topics.
*"Stream" (Transifex), "Kanal" (KDE IRC documentation, various
small German forums)*
_"Stream" (Transifex), "Kanal" (KDE IRC documentation, various
small German forums)_
- Topic - **Thema**
*(Gmail - for email subjects, Transifex)*
_(Gmail - for email subjects, Transifex)_
- Invite-Only Stream - **Geschlossener Stream**
For users to be able to join to an "invite-only" stream, they must have been
invited by some user in this stream. This type of stream is equivalent to
Facebook's "closed" groups, which in turn translates to "geschlossen" in German.
This translation seems to be appropriate, for example [Linguee](
https://www.linguee.de/englisch-deutsch/uebersetzung/invite-only.html)
This translation seems to be appropriate, for example [Linguee](https://www.linguee.de/englisch-deutsch/uebersetzung/invite-only.html)
search returns only paraphrases of this term.
*"Geschlossener Stream" (Transifex), "Geschlossene Gruppe" (Facebook),
paraphrases (Linguee)*
_"Geschlossener Stream" (Transifex), "Geschlossene Gruppe" (Facebook),
paraphrases (Linguee)_
- Public Stream - **Öffentlicher Stream**
@@ -183,14 +180,14 @@ While some might find this direct translation a tad long, the alternative
"Offener Stream" can be ambiguous - especially users who are inexperienced
with Zulip could think of this as streams that are online.
*"Öffentlicher Stream" (Transifex)*
_"Öffentlicher Stream" (Transifex)_
- Bot - **Bot**
Not only is "bot" a short and easily memorable term, it is also widely used
in German technology magazines, forums, etc.
*"Bot" (Transifex, Heise, Die Zeit)*
_"Bot" (Transifex, Heise, Die Zeit)_
- Integration - **Integration**
@@ -200,34 +197,34 @@ use "Integrationen" instead of "Integrations" when speaking of multiple
integrations in German. There aren't many German sources available for this
translation, but "Integration" has the same meaning in German and English.
*"Integration/-en" (Transifex)*
_"Integration/-en" (Transifex)_
- Notification - **Benachrichtigung**
Nice and easy. Other translations for "notification" like
"Erwähnung", "Bescheid" or "Notiz" don't fit here.
*"Benachrichtigung" (Facebook, Gmail, Transifex, Wikipedia)*
_"Benachrichtigung" (Facebook, Gmail, Transifex, Wikipedia)_
- Alert Word - **Signalwort**
This one is tricky, since one might initially think of "Alarmwort" as a proper
translation. "Alarm", however, has a negative connotation, people link it to
unpleasant events. "Signal", on the other hand, is neutral, just like
"alert word". Nevertheless, [Linguee](
https://www.linguee.de/deutsch-englisch/search?source=auto&query=alert+word)
"alert word". Nevertheless, [Linguee](https://www.linguee.de/deutsch-englisch/search?source=auto&query=alert+word)
shows that some websites misuse "Alarm" for the translation.
*"Signalwort" (Transifex), "Wort-Alarm" (Linguee)*
_"Signalwort" (Transifex), "Wort-Alarm" (Linguee)_
- View - **View** (Developer documentation)
Since this is a Zulip-specific term for
> every path that the Zulip server supports (doesn’t show a 404 page for),
and there is no German equivalent, talking of "Views" is preferable in the
developer documentation and makes it easier to rely on parts of the German
*and* parts of the English documentation.
_and_ parts of the English documentation.
- View - **Ansicht** (User documentation)
@@ -237,13 +234,13 @@ For the user documentation, we want to use "Ansicht" instead of "view", as
does not emphasize the developing aspects of views (in contrast to anglicisms,
which Germans often link to IT-related definitions).
*"Ansicht" (Transifex)*
_"Ansicht" (Transifex)_
- Home - **Startseite**
Nice and easy. "Zuhause" obviously doesn't fit here ;).
*"Startseite" (Facebook, Transifex)*
_"Startseite" (Facebook, Transifex)_
- Emoji - **Emoji**
@@ -251,8 +248,7 @@ Nice and easy. "Zuhause" obviously doesn't fit here ;).
"Bildschriftzeichen" (which exists!) would sound stiff and outdated. "Emoticon"
works as well, but is not that common in German.
*"Emoji" (Facebook, WhatsApp), "Emoticon" (Google+)*
_"Emoji" (Facebook, WhatsApp), "Emoticon" (Google+)_
## Phrases (Ausdrücke)
@@ -260,7 +256,7 @@ works as well, but is not that common in German.
This translation is unambiguous.
*"Deabonnieren" (YouTube, Transifex)*
_"Deabonnieren" (YouTube, Transifex)_
- Narrow to - **Begrenzen auf**
@@ -272,14 +268,14 @@ would be too long for many labels, the infinitive "begrenzen auf" is preferable.
"einschränken auf" sounds equally good, but Transifex shows more use cases for
"begrenzen auf".
*"Schränke auf ... ein." (Transifex) "Begrenze auf ... ." (Transifex)*
_"Schränke auf ... ein." (Transifex) "Begrenze auf ... ." (Transifex)_
- Filter - **Filtern**
A direct translation is fine here. Watch out to to use the infinitive instead
of the imperative, e.g. "Nachrichten filtern" instead of "Filtere Nachrichten".
*"Filtern" (Thunderbird, LinkedIn)*
_"Filtern" (Thunderbird, LinkedIn)_
- Mute/Unmute - **Stummschalten/Lautschalten**
@@ -289,11 +285,11 @@ preferable due to its brevity.
- Deactivate/Reactivate - **Deaktivieren/Reaktivieren**
*"Deaktivieren/Reaktivieren" (Transifex)*
_"Deaktivieren/Reaktivieren" (Transifex)_
- Search - **Suchen**
*"Suchen" (YouTube, Google, Facebook, Transifex)*
_"Suchen" (YouTube, Google, Facebook, Transifex)_
- Pin/Unpin - **Anpinnen/Loslösen**
@@ -301,17 +297,17 @@ While "pinnen" is shorter than "anpinnen", "anpinnen" sweeps any amiguity out of
the way. This term is not used too often on Zulip, so the length shouldn't be a
problem.
*"Anpinnen/Ablösen" (Transifex), "Pinnen" (Pinterest)*
_"Anpinnen/Ablösen" (Transifex), "Pinnen" (Pinterest)_
- Mention/@mention - **Erwähnen/"@-Erwähnen**
Make sure to say "@-erwähnen", but "die @-Erwähnung" (capitalized).
*"Erwähnen/@-Erwähnen" (Transifex)*
_"Erwähnen/@-Erwähnen" (Transifex)_
- Invalid - **Ungültig**
*"Ungültig" (Transifex)*
_"Ungültig" (Transifex)_
- Customization - **Anpassen**
@@ -322,21 +318,21 @@ the infinitive form "anpassen".
"Ich möchte" is the polite form of "Ich will".
*"Ich möchte" - (Transifex, general sense of politeness)*
_"Ich möchte" - (Transifex, general sense of politeness)_
- User - **Nutzer**
"Benutzer" would work as well, but "Nutzer" is shorter and more commonly
used in web applications.
*"Nutzer" (Facebook, Gmail), "Benutzer" (Transifex)*
_"Nutzer" (Facebook, Gmail), "Benutzer" (Transifex)_
- Person/People - Nutzer/Personen
We use "Personen" instead of plural "Nutzer" for "people", as "Nutzer" stays
the same in plural.
*"Nutzer/Personen" (Transifex)*
_"Nutzer/Personen" (Transifex)_
## Other (Verschiedenes)
@@ -347,11 +343,11 @@ informal language. If you would like to read more about the reasoning behind
this, refer to the [general notes](#formal-or-informal) for
translating German.
*"Du" (Google, Facebook), "Sie" (Transifex)*
_"Du" (Google, Facebook), "Sie" (Transifex)_
- We - **Wir** (rarely used)
German guides don't use "wir" very often - they tend to reformulate the
phrases instead.
*"Wir" (Google, Transifex)*
_"Wir" (Google, Transifex)_

19
docs/translating/hindi.md
View File

@@ -2,13 +2,13 @@
Use informal Hindi for translation:
- Informal "you" (*तु*) instead of formal form *आप*. Many top software
- Informal "you" (_तु_) instead of formal form _आप_. Many top software
companies (e.g. Google) use the informal one, because it's much more common in
the daily language and avoids making translations look like they were written
by machines.
- Imperative, active, and continuous verbs, e.g. *manage streams* -
*चैनल प्रबंधित करें*, not *चैनल प्रबंधन*.
- Imperative, active, and continuous verbs, e.g. _manage streams_ -
_चैनल प्रबंधित करें_, not _चैनल प्रबंधन_.
- Warm and friendly phrasing whenever appropriate.
@@ -24,6 +24,7 @@ with other Hindi speakers in the community. It's all about making
Zulip friendly and usable.
## Terms(शर्तें)
- Message - **संदेश**
- Private message (PM) - **निजी संदेश**
- Stream - **धारा**: the use of the literal Hindi word for stream
@@ -35,8 +36,8 @@ Zulip friendly and usable.
- Bot - **बॉट**
- Integration - **एकीकरण**
- Notification - **अधिसूचना**
- Alert word - **सतर्क शब्द**: this is only *alert*. Nonetheless, adding *word* may
make the term confusing (something like *danger!* could be a "चेतावनी के शब्द" as well).
- Alert word - **सतर्क शब्द**: this is only _alert_. Nonetheless, adding _word_ may
make the term confusing (something like _danger!_ could be a "चेतावनी के शब्द" as well).
Google Alerts uses "सतर्क शब्द" in its Hindi translation.
- View - **राय**
- Filter - **छानना**: as used with narrowing (see below).
@@ -44,9 +45,10 @@ Zulip friendly and usable.
- Emoji - **इमोजी**
## Phrases (वाक्यांशों)
- Subscribe/Unsubscribe to a stream - **एक धारा में सदस्यता लें/सदस्यता समाप्त करें**
- Narrow to - **अकेले फ़िल्टर करें**: this is *filter only*, because there's no other
word that's common enough in Hindi for *to narrow*.
- Narrow to - **अकेले फ़िल्टर करें**: this is _filter only_, because there's no other
word that's common enough in Hindi for _to narrow_.
- Mute/Unmute - **शांत/अशांत**
- Deactivate/Reactivate - **निष्क्रिय करें / पुन: सक्रिय करें**
- Search - **खोज करें/ढूंढे**
@@ -57,10 +59,11 @@ Zulip friendly and usable.
- I want - **मुझे चाहिए**
- User - **उपयोगकर्ता**
- Person/People - **व्यक्ति/लोग**: "लोग" is the correct plural for
"व्यक्ति", but when talking of *लोग* referring to it as a crowd, we use
"व्यक्ति", but when talking of _लोग_ referring to it as a crowd, we use
"भीड़" instead.
## Others(अन्य)
- You - **तुम**: also "आप" if it's in plural.
- We - **हम**
- Message table - **संदेश बोर्ड**

14
docs/translating/internationalization.md
View File

@@ -272,6 +272,7 @@ If you are passing a translated string to a Handlebars partial, you can use:
The syntax for block strings or strings containing variables is:
<!-- The html+handlebars lexer fails to lex the single braces. -->
```text
{{#tr}}
Block of English text.
@@ -282,7 +283,7 @@ The syntax for block strings or strings containing variables is:
{{/tr}}
```
Just like in JavaScript code, variables are enclosed in *single*
Just like in JavaScript code, variables are enclosed in _single_
braces (rather than the usual Handlebars double braces). Unlike in
JavaScript code, variables are automatically escaped by our Handlebars
helper.
@@ -331,13 +332,12 @@ hostname = https://www.transifex.com
This basically identifies you as a Transifex user, so you can access your
organizations from the command line.
[Jinja2]: http://jinja.pocoo.org/
[Handlebars]: https://handlebarsjs.com/
[jinja2]: http://jinja.pocoo.org/
[handlebars]: https://handlebarsjs.com/
[trans]: http://jinja.pocoo.org/docs/dev/templates/#i18n
[FormatJS]: https://formatjs.io/
[ICU MessageFormat]: https://formatjs.io/docs/intl-messageformat
[formatjs]: https://formatjs.io/
[icu messageformat]: https://formatjs.io/docs/intl-messageformat
[helpers]: https://handlebarsjs.com/guide/block-helpers.html
[Transifex]: https://transifex.com
[transifex]: https://transifex.com
[transifexrc]: https://docs.transifex.com/client/client-configuration#transifexrc
[html-templates]: ../subsystems/html-css.html#html-templates

69
docs/translating/polish.md
View File

@@ -2,22 +2,22 @@
Use semi-formal Polish for translation, some specifics:
- Informal "you" (*ty*) instead of more formal approaches (e.g. plural
"you" (*wy*), using any formal titles like *Państwo*, *Pan/Pani*).
- Informal "you" (_ty_) instead of more formal approaches (e.g. plural
"you" (_wy_), using any formal titles like _Państwo_, _Pan/Pani_).
- Gender-neutral forms of verbs, e.g. *unsubscribed* - *odsubskrybowano*,
not *odsubskrybowałeś".
- Gender-neutral forms of verbs, e.g. _unsubscribed_ - _odsubskrybowano_,
not \*odsubskrybowałeś".
- Imperative, active and continuous verbs, e.g. *manage streams* -
*zarządzaj kanałami*, not *zarządź kanałami*.
- Imperative, active and continuous verbs, e.g. _manage streams_ -
_zarządzaj kanałami_, not _zarządź kanałami_.
- Not using reflexive *się*, e.g. *log out* would be simply *wyloguj*,
not *wyloguj się*.
- Not using reflexive _się_, e.g. _log out_ would be simply _wyloguj_,
not _wyloguj się_.
- Warm and friendly phrasing whenever appropriate.
- No slang or regional phrases that could be unclear or too informal,
e.g. *zajawka*.
e.g. _zajawka_.
- Consistent usage of Zulip-specific terms and common verbs for
actions, even if it means repeating - this is one of the key aspects
@@ -40,11 +40,12 @@ Zulip friendly and usable.
## Special terms used in Zulip
**alert word**: powiadomienie, "ostrzeżenie" could mean something negative
and alert words in Zulip are used to help users find relevant content
and alert words in Zulip are used to help users find relevant content
example:
You can set your own alert words for Zulip messages.
> Możesz ustawić powiadomienia dla wybranych fraz w Zulipie.
**All messages**: wszystkie wiadomości
@@ -52,6 +53,7 @@ You can set your own alert words for Zulip messages.
example:
You can see all messages in unmuted streams and topics with "All messages".
> Możesz zobaczyć pełną listę wiadomości poprzez widok "Wszystkie wiadomości".
**bot**: bot
@@ -59,21 +61,24 @@ You can see all messages in unmuted streams and topics with "All messages".
example:
You can add bots to your Zulip.
> Możesz dodać boty do swojego Zulipa.
**customization**: personalizacja, *kastomizacja* could be too awkward
and *dostosowanie do potrzeb klienta* is too long
**customization**: personalizacja, _kastomizacja_ could be too awkward
and _dostosowanie do potrzeb klienta_ is too long
example:
You can personalize Zulip in many ways, e.g. by pinning certain streams.
> Możesz spersonalizować Zulipa na wiele sposobów, np. przypinając niektóre kanały.
**emoji**: emoji, both in singular and plural, *ikona emoji* is a pleonasm
**emoji**: emoji, both in singular and plural, _ikona emoji_ is a pleonasm
example:
Zulip supports emoji both in messages and as reactions.
> Zulip wspiera używanie emoji w wiadomościach i jako reakcje.
**filter**: filtr (noun) and filtrowanie (verb)
@@ -81,14 +86,16 @@ Zulip supports emoji both in messages and as reactions.
example:
You can filter the messages by searching for relevant terms.
> Możesz przefiltrować wiadomości poprzez wyszukiwanie.
**group PM**: czat grupowy, different from *wiadomość* since the usage
of *czat grupowy* seems more common
**group PM**: czat grupowy, different from _wiadomość_ since the usage
of _czat grupowy_ seems more common
example:
You can start a group PM with users in your organization.
> Możesz rozpocząć czat grupowy z użytkownikami w Twojej organizacji.
**integration**: integracja
@@ -96,6 +103,7 @@ You can start a group PM with users in your organization.
example:
Zulip supports multiple third-party integrations.
> Zulip wspiera wiele zewnętrznych integracji.
**I want**: chcę
@@ -103,6 +111,7 @@ Zulip supports multiple third-party integrations.
example:
I want to change my password.
> Chcę zmienić hasło.
**invalid**: niepoprawny/a
@@ -110,6 +119,7 @@ I want to change my password.
example:
Invalid command.
> Niepoprawna instrukcja.
**me**: me, no translation since it's used as `/me`
@@ -117,6 +127,7 @@ Invalid command.
example:
You can use `/me` to write a reaction message.
> Możesz napisać wiadomość w formie komentarza poprzez użycie `me`.
**mention**: oznaczenie (noun) and oznaczyć (verb)
@@ -124,6 +135,7 @@ You can use `/me` to write a reaction message.
example:
You can mention other Zulip users by using @.
> Możesz oznaczyć innych użytkowników Zulipa używając @.
**message**: wiadomość
@@ -131,6 +143,7 @@ You can mention other Zulip users by using @.
example:
You got a new message.
> Masz nową wiadomość.
**message table**: lista wiadomości
@@ -138,6 +151,7 @@ You got a new message.
example:
The middle column in Zulip is a message table of all messages in current narrow.
> Środkowa kolumna w Zulipie zawiera listę wiadomości w wybranym widoku.
**muting a stream/topic**: wyciszenie kanału/wątku
@@ -146,6 +160,7 @@ The middle column in Zulip is a message table of all messages in current narrow.
example:
You can mute any topic in Zulip through the left-side panel.
> Możesz wyciszyć dowolny wątek w Zulipie używając menu kontekstowego po lewej.
**narrow**: widok (noun) and zawęzić (verb)
@@ -153,6 +168,7 @@ You can mute any topic in Zulip through the left-side panel.
example:
You can narrow the messages to any stream, topic or search results.
> Możesz zawęzić wiadomości do wybranego kanału, wątku lub wyników wyszukiwania.
**notification**: powiadomienie
@@ -160,6 +176,7 @@ You can narrow the messages to any stream, topic or search results.
example:
Turn on notifications.
> Włącz powiadomienia.
**person**: osoba, najczęściej użytkownik
@@ -170,6 +187,7 @@ Turn on notifications.
example:
You can pin streams in the left-side panel.
> Możesz przypiąć wybrane kanały w menu kontekstowym po lewej.
**private message**: prywatna wiadomość
@@ -177,6 +195,7 @@ You can pin streams in the left-side panel.
example:
You can send a private message to other users in your organization.
> Możesz wysłać prywatną wiadomość do innych użytkowników w Twojej organizacji.
**PM**: PM, translation could be confusing
@@ -184,6 +203,7 @@ You can send a private message to other users in your organization.
example:
Private messages are often abbreviated to PM.
> Prywatne wiadomości są też nazywane PMami, od angielskiego "private message".
**private stream**: prywatny kanał
@@ -192,6 +212,7 @@ Private messages are often abbreviated to PM.
example:
Join a private stream.
> Dołącz do prywatnego kanału.
**search**: wyszukiwanie (noun) and wyszukaj (verb)
@@ -200,8 +221,9 @@ example:
Zulip allows you to search messages in all streams and topics you are
subscribed to.
> Zulip pozwala na wyszukiwanie wiadomości we wszystkich subskrybowanych kanałach
i wątkach.
> i wątkach.
**starred message**: oznaczona wiadomość (noun)
**star**: oznaczyć
@@ -209,25 +231,29 @@ subscribed to.
example:
You starred this message.
> Ta wiadomość została oznaczona.
**stream**: kanał, similar to a tv channel - *strumień* sounds a bit artificial
**stream**: kanał, similar to a tv channel - _strumień_ sounds a bit artificial
example:
You can create new streams in Zulip.
> Możesz tworzyć nowe kanały w Zulipie.
**subscribing to a stream**: (za)subskrybowanie kanału (noun) and
(za)subskrybować kanał (verb), perfective form depending on the
context
(za)subskrybować kanał (verb), perfective form depending on the
context
examples:
Subscribe to a stream.
> Zasubskrybuj kanał.
You are not subscribed to this stream.
> Nie subskrybujesz tego kanału.
**topic**: wątek
@@ -235,14 +261,16 @@ You are not subscribed to this stream.
example:
Add a new topic.
> Dodaj nowy wątek.
**unsubscribing from a stream**: odsubskrybowanie kanału (noun) and
odsubskrybować kanał (verb)
odsubskrybować kanał (verb)
example:
You have unsubscribed from this stream.
> Odsubskrybowano kanał.
**user**: użytkownik
@@ -250,6 +278,7 @@ You have unsubscribed from this stream.
example:
Zulip supports an unlimited number of users in an organization.
> Zulip nie limituje liczby użytkowników w organizacji.
**view**: widok, see: **narrow**

29
docs/translating/spanish.md
View File

@@ -2,15 +2,15 @@
Use informal Spanish for translation:
- Informal "you" (*tú*) instead of formal form *usted*. Many top software
- Informal "you" (_tú_) instead of formal form _usted_. Many top software
companies (e.g. Google) use the informal one, because it's much more common in
the daily language and avoids making translations look like they were written
by machines.
- Imperative, active, and continuous verbs, e.g. *manage streams* -
*gestionar canales*, not *gestión de canales*.
- Imperative, active, and continuous verbs, e.g. _manage streams_ -
_gestionar canales_, not _gestión de canales_.
- Not using reflexive *se* e.g. *log out* should be *salir*, not *salirse*,
- Not using reflexive _se_ e.g. _log out_ should be _salir_, not _salirse_,
whenever the infinitive form is possible without making the translation
awkward.
@@ -32,26 +32,27 @@ with other Spanish speakers in the community. It's all about making
Zulip friendly and usable.
## Términos
- Message - **Mensaje**
- Private message (PM) - **Mensaje privado (MP)**
- Group PM - **mensaje privado grupal**: even though "MP grupal" is the most
precise translation, preferably don't use that one. Many users may not
associate "MP" with *private message* in a group context. Better use it
associate "MP" with _private message_ in a group context. Better use it
without abbreviations.
- Stream - **Canal**: the use of the literal Spanish word for stream
"Flujo" is very confusing and not the correct metaphor for Spanish
speaking people. The correct term would be "canal" (*channel*).
speaking people. The correct term would be "canal" (_channel_).
- Topic - **Tema**
- Private/invite-only stream - **Canal privado/limitado por invitación**: (lit.
*channel limited by invitation*)
_channel limited by invitation_)
- Public stream - **Canal público**
- Bot - **Bot**
- Embedded bot - **Bot integrado**
- Interactive bot - **Bot interactivo**
- Integration - **Integración**
- Notification - **Notificación**
- Alert word - **Alerta**: this is only *alert*. Nonetheless, adding *word* may
make the term confusing (something like *danger!* could be a "palabra de
- Alert word - **Alerta**: this is only _alert_. Nonetheless, adding _word_ may
make the term confusing (something like _danger!_ could be a "palabra de
alerta" as well). Google Alerts uses "alerta" in its Spanish translation.
- View - **Vista**
- Filter - **Filtro**: as used with narrowing (see below).
@@ -62,23 +63,25 @@ Zulip friendly and usable.
- Endpoint - **Endpoint**
## Frases
- Subscribe/Unsubscribe to a stream - **Suscribir a/Desuscribir de un canal**
- Narrow to - **Filtrar solo**: this is *filter only*, because there's no other
word that's common enough in Spanish for *to narrow* except for "filtrar".
- Narrow to - **Filtrar solo**: this is _filter only_, because there's no other
word that's common enough in Spanish for _to narrow_ except for "filtrar".
- Mute/Unmute - **Silenciar/No silenciar**
- Deactivate/Reactivate - **Desactivar/Reactivar**
- Search - **Buscar**
- Pin - **Fijar** (lit. *to fixate*)
- Pin - **Fijar** (lit. _to fixate_)
- Mention/@mention - **Mencionar/@mención**
- Invalid - **Inválido**
- Customization - **Personalización**
- I want - **Yo quiero**
- User - **Usuario**
- Person/People - **Persona/Personas**: "personas" is the correct plural for
"person", but when talking of *people* referring to it as a crowd, we use
"person", but when talking of _people_ referring to it as a crowd, we use
"gente" instead.
## Otros
- You - **Tú**: also "vosotros" if it's in plural.
- We - **Nosotros**
- Message table - **Tablón de mensajes**

35
docs/translating/translating.md
View File

@@ -21,13 +21,13 @@ These are the steps you should follow if you want to help translate
Zulip:
1. Join [#translation][translation-stream] in the [Zulip development
community server](../contributing/chat-zulip-org.md), and say hello.
That stream is also the right place for any questions, updates on your
progress, reporting problematic strings, etc.
community server](../contributing/chat-zulip-org.md), and say hello.
That stream is also the right place for any questions, updates on your
progress, reporting problematic strings, etc.
1. Sign up for [Transifex](https://www.transifex.com) and ask to join the [Zulip
project on Transifex](https://www.transifex.com/zulip/zulip/), requesting access
to any languages that you'd like to contribute to (or add new ones).
project on Transifex](https://www.transifex.com/zulip/zulip/), requesting access
to any languages that you'd like to contribute to (or add new ones).
```eval_rst
.. note::
@@ -44,6 +44,7 @@ to any languages that you'd like to contribute to (or add new ones).
1. Translate the strings for your language in Transifex. Zulip has
several resource files:
- `mobile.json` is for the iOS/Android mobile apps.
- `desktop.json` is for the parts of the Zulip desktop apps that
are not shared with the Zulip webapp.
@@ -90,7 +91,7 @@ Some useful tips for your translating journey:
the [Zulip development community server](../contributing/chat-zulip-org.md).
- If there are multiple possible translations for a term, search for it in
the *Concordance* tool (the button with a magnet in the top right corner).
the _Concordance_ tool (the button with a magnet in the top right corner).
It will show if anyone translated that term before, so we can achieve good
consistency with all the translations, no matter who makes them.
@@ -109,11 +110,11 @@ if setting one up is a problem for you, ask in chat.zulip.org and we
can usually just deploy the latest translations there.
- First, download the updated resource files from Transifex using the
`tools/i18n/sync-translations` command (it will require some [initial
setup](../translating/internationalization.html#transifex-cli-setup)). This
command will download the resource files from Transifex and replace
your local resource files with them, and then compile them. You can
now test your translation work in the Zulip UI.
`tools/i18n/sync-translations` command (it will require some [initial
setup](../translating/internationalization.html#transifex-cli-setup)). This
command will download the resource files from Transifex and replace
your local resource files with them, and then compile them. You can
now test your translation work in the Zulip UI.
There are a few ways to see your translations in the Zulip UI:
@@ -182,15 +183,15 @@ properly capitalized in a way consistent with how Zulip does
capitalization in general. This means that:
- The first letter of a sentence or phrase should be capitalized.
- Correct: "Manage streams"
- Incorrect: "Manage Streams"
- Correct: "Manage streams"
- Incorrect: "Manage Streams"
- All proper nouns should be capitalized.
- Correct: "This is Zulip"
- Incorrect: "This is zulip"
- Correct: "This is Zulip"
- Incorrect: "This is zulip"
- All common words like URL, HTTP, etc. should be written in their
standard forms.
- Correct: "URL"
- Incorrect: "Url"
- Correct: "URL"
- Incorrect: "Url"
The Zulip test suite enforces these capitalization guidelines in the
webapp codebase [in our test

2
docs/tutorials/life-of-a-request.md
View File

@@ -99,7 +99,7 @@ for its API. This means that we use:
useful to check a link without downloading a potentially large link
- OPTIONS (handled automatically, see more below)
Of these, PUT, DELETE, HEAD, OPTIONS, and GET are *idempotent*, which
Of these, PUT, DELETE, HEAD, OPTIONS, and GET are _idempotent_, which
means that we can send the request multiple times and get the same
state on the server. You might get a different response after the first
request, as we like to give our clients an error so they know that no

71
docs/tutorials/new-feature-tutorial.md
View File

@@ -34,6 +34,7 @@ This tutorial will walk through adding a new feature to a Realm (an
organization in Zulip). The following files are involved in the process:
**Backend**
- `zerver/models.py`: Defines the database model.
- `zerver/views/realm.py`: The view function that implements the API endpoint
for editing realm objects.
@@ -42,20 +43,23 @@ organization in Zulip). The following files are involved in the process:
consistent and correct.
**Frontend**
- `static/templates/settings/organization_permissions_admin.hbs`: defines
the structure of the admin permissions page (checkboxes for each organization
permission setting).
the structure of the admin permissions page (checkboxes for each organization
permission setting).
- `static/js/settings_org.js`: handles organization setting form submission.
- `static/js/server_events_dispatch.js`: handles events coming from the server
(ex: pushing an organization change to other open browsers and updating
the application's state).
**Backend testing**
- `zerver/tests/test_realm.py`: end-to-end API tests for updating realm settings.
- `zerver/tests/test_events.py`: tests for possible race bugs in the
zerver/lib/events.py implementation.
**Frontend testing**
- `frontend_tests/puppeteer_tests/admin.ts`: end-to-end tests for the organization
admin settings pages.
- `frontend_tests/node_tests/dispatch.js`
@@ -79,8 +83,7 @@ to learn more about creating and applying database migrations.
**Test your changes:** Once you've run the migration, flush memcached
on your development server (`./scripts/setup/flush-memcached`) and then
[restart the development server](
../development/remote.html?highlight=tools%2Frun-dev.py#running-the-development-server)
[restart the development server](../development/remote.html?highlight=tools%2Frun-dev.py#running-the-development-server)
to avoid interacting with cached objects.
### Backend changes
@@ -152,14 +155,12 @@ documentation Zulip has, see [Documentation](../documentation/overview.md).
This example describes the process of adding a new setting to Zulip: a
flag that allows an admin to require topics on stream messages (the default
behavior is that topics can have no subject). This flag is an
actual Zulip feature. You can review [the original commit](
https://github.com/zulip/zulip/pull/5660/commits/aeeb81d3ff0e0cc201e891cec07e1d2cd0a2060d)
actual Zulip feature. You can review [the original commit](https://github.com/zulip/zulip/pull/5660/commits/aeeb81d3ff0e0cc201e891cec07e1d2cd0a2060d)
in the Zulip repo. (This commit displays the work of setting up a checkbox
for the feature on the admin settings page, communicating and saving updates
to the setting to the database, and updating the state of the application
after the setting is updated. For the code that accomplishes the underlying
task of requiring messages to have a topic, you can [view this commit](
https://github.com/zulip/zulip/commit/90e2f5053f5958b44ea9b2362cadcb076deaa975).)
task of requiring messages to have a topic, you can [view this commit](https://github.com/zulip/zulip/commit/90e2f5053f5958b44ea9b2362cadcb076deaa975).)
### Update the model
@@ -167,7 +168,7 @@ First, update the database and model to store the new setting. Add a new
boolean field, `mandatory_topics`, to the Realm model in
`zerver/models.py`.
``` diff
```diff
# zerver/models.py
class Realm(models.Model):
@@ -184,7 +185,7 @@ dictionary, where the key is the name of the realm field and the value
is the field's type. Add the new field to the `property_types`
dictionary.
``` diff
```diff
# zerver/models.py
class Realm(models.Model)
@@ -203,10 +204,10 @@ dictionary.
logic and thus cannot use this framework. For example:
- The realm `authentication_methods` attribute is a bitfield and needs
additional code for validation and updating.
additional code for validation and updating.
- The `allow_message_editing` and `message_content_edit_limit_seconds`
fields depend on one another, so they are also handled separately and
not included in `property_types`.
fields depend on one another, so they are also handled separately and
not included in `property_types`.
When creating a realm property that is not a boolean, Text or
integer field, or when adding a field that is dependent on other fields,
@@ -229,6 +230,7 @@ is helpful.
Apply the migration using Django's `migrate` command: `./manage.py migrate`.
Output:
```console
shell $ ./manage.py migrate
Operations to perform:
@@ -244,8 +246,7 @@ Running migrations:
```
Once you've run the migration, restart memcached on your development
server (`/etc/init.d/memcached restart`) and then [restart the development server](
../development/remote.html?highlight=tools%2Frun-dev.py#running-the-development-server)
server (`/etc/init.d/memcached restart`) and then [restart the development server](../development/remote.html?highlight=tools%2Frun-dev.py#running-the-development-server)
to avoid interacting with cached objects.
### Handle database interactions
@@ -255,7 +256,7 @@ Like typical apps, we will need our backend to update the database and
send some response to the client that made the request.
Beyond that, we need to orchestrate notifications about the setting change
to *other* clients (or other users, if you will). Clients
to _other_ clients (or other users, if you will). Clients
find out about settings through two closely related code paths. When a client
first contacts the server, the server sends the client its
initial state. Subsequently, clients subscribe to "events," which can
@@ -414,7 +415,7 @@ You'll need to add a parameter for the new field to the `update_realm`
function in `zerver/views/realm.py` (and add the appropriate mypy type
annotation).
``` diff
```diff
# zerver/views/realm.py
def update_realm(
@@ -516,7 +517,7 @@ template.
Then add the new form control in `static/js/admin.js`.
``` diff
```diff
// static/js/admin.js
function _setup_page() {
@@ -536,27 +537,27 @@ The JavaScript code for organization settings and permissions can be found in
In frontend, we have split the `property_types` into three objects:
- `org_profile`: This contains properties for the "organization
profile" settings page.
profile" settings page.
- `org_settings`: This contains properties for the "organization
settings" page. Settings belonging to this section generally
decide what features should be available to a user like deleting a
message, message edit history etc. Our `mandatory_topics` feature
belongs in this section.
settings" page. Settings belonging to this section generally
decide what features should be available to a user like deleting a
message, message edit history etc. Our `mandatory_topics` feature
belongs in this section.
- `org_permissions`: This contains properties for the "organization
permissions" section. These properties control security controls
like who can join the organization and whether normal users can
create streams or upload custom emoji.
permissions" section. These properties control security controls
like who can join the organization and whether normal users can
create streams or upload custom emoji.
Once you've determined whether the new setting belongs, the next step
is to find the right subsection of that page to put the setting
in. For example in this case of `mandatory_topics` it will lie in
"Other settings" (`other_settings`) subsection.
*If you're not sure in which section your feature belongs, it's
_If you're not sure in which section your feature belongs, it's
better to discuss it in the [community](https://chat.zulip.org/)
before implementing it.*
before implementing it._
Note that some settings, like `realm_msg_edit_limit_setting`,
require special treatment, because they don't match the common
@@ -565,14 +566,14 @@ such input elements with those in `page_params`, so we have to
manually handle such situations in a couple key functions:
- `settings_org.get_property_value`: This processes the property name
when it doesn't match a corresponding key in `page_params`, and
returns the current value of that property, which we can use to
compare and set the values of corresponding DOM element.
when it doesn't match a corresponding key in `page_params`, and
returns the current value of that property, which we can use to
compare and set the values of corresponding DOM element.
- `settings_org.update_dependent_subsettings`: This handles settings
whose value and state depend on other elements. For example,
`realm_waiting_period_threshold` is only shown for with the right
state of `realm_waiting_period_setting`.
whose value and state depend on other elements. For example,
`realm_waiting_period_threshold` is only shown for with the right
state of `realm_waiting_period_setting`.
Finally, update `server_events_dispatch.js` to handle related events coming from
the server. There is an object, `realm_settings`, in the function
@@ -589,7 +590,7 @@ setting has changed, your function should be referenced in the
`realm_settings` of `server_events_dispatch.js`. See for example
`settings_emoji.update_custom_emoji_ui`.
``` diff
```diff
// static/js/server_events_dispatch.js
function dispatch_normal_event(event) {

93
docs/tutorials/reading-list.md
View File

@@ -6,87 +6,86 @@ learning new skills, or polishing the ones you already have.
The topics cover a wide variety of topics, from basic Python coding to
general developing guidelines.
Feel free to create a pull request in Zulip's [GitHub repository](
https://github.com/zulip/zulip/) with any interesting books, articles or
Feel free to create a pull request in Zulip's [GitHub repository](https://github.com/zulip/zulip/) with any interesting books, articles or
videos you would like to see in this list.
Some titles have been shortened for organizational purposes.
## General programming/IT
*Book* - [Clean Code: A Handbook of Agile Software Craftsmanship](https://www.amazon.com/Clean-Code-Handbook-Software-Craftsmanship/dp/0132350882) *(Not free!)*
_Book_ - [Clean Code: A Handbook of Agile Software Craftsmanship](https://www.amazon.com/Clean-Code-Handbook-Software-Craftsmanship/dp/0132350882) _(Not free!)_
*Books* - [Free programming books list](https://github.com/vhf/free-programming-books)
_Books_ - [Free programming books list](https://github.com/vhf/free-programming-books)
*Blog* - [Free Code Camp blog](https://medium.freecodecamp.com)
_Blog_ - [Free Code Camp blog](https://medium.freecodecamp.com)
*Blog* - [Idle Words talks transcripts](https://idlewords.com/talks)
_Blog_ - [Idle Words talks transcripts](https://idlewords.com/talks)
*Tutorial* - [HTTP Can Do That?!, by Sumana Harihareswara (PyCon 2016)](https://youtu.be/HsLrXt2l-kg)
_Tutorial_ - [HTTP Can Do That?!, by Sumana Harihareswara (PyCon 2016)](https://youtu.be/HsLrXt2l-kg)
*Video* - [Minimum Viable Documentation, by Matthew Lyon (WriteTheDocs 2014)](https://youtu.be/bEZcodengwk)
_Video_ - [Minimum Viable Documentation, by Matthew Lyon (WriteTheDocs 2014)](https://youtu.be/bEZcodengwk)
*Video* - [NoOps, by Kelsey Hightower (DepOpsDays 2016)](https://youtu.be/ajT90pC3ris)
_Video_ - [NoOps, by Kelsey Hightower (DepOpsDays 2016)](https://youtu.be/ajT90pC3ris)
*Video* - [The mind behind Linux (TED interview)](https://youtu.be/o8NPllzkFhE)
_Video_ - [The mind behind Linux (TED interview)](https://youtu.be/o8NPllzkFhE)
*Tutorial* - [Learn code the hard way](https://learncodethehardway.org)
_Tutorial_ - [Learn code the hard way](https://learncodethehardway.org)
*Tutorial* - [What happens when...](https://github.com/alex/what-happens-when)
_Tutorial_ - [What happens when...](https://github.com/alex/what-happens-when)
*Article* - [An Interview With Linus Torvalds](https://techcrunch.com/2012/04/19/an-interview-with-millenium-technology-prize-finalist-linus-torvalds)
_Article_ - [An Interview With Linus Torvalds](https://techcrunch.com/2012/04/19/an-interview-with-millenium-technology-prize-finalist-linus-torvalds)
*Article* - [Effective Learning Strategies for Programmers](https://akaptur.com/blog/2015/10/10/effective-learning-strategies-for-programmers/)
_Article_ - [Effective Learning Strategies for Programmers](https://akaptur.com/blog/2015/10/10/effective-learning-strategies-for-programmers/)
*Article* - [Readme Driven Development](https://tom.preston-werner.com/2010/08/23/readme-driven-development.html)
_Article_ - [Readme Driven Development](https://tom.preston-werner.com/2010/08/23/readme-driven-development.html)
*Article* - [Systematic Debugging](https://akaptur.com/blog/2013/07/24/systematic-debugging)
_Article_ - [Systematic Debugging](https://akaptur.com/blog/2013/07/24/systematic-debugging)
*Paper* - [Floating-Point Arithmetic](https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html)
_Paper_ - [Floating-Point Arithmetic](https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html)
## Python
*Video* - [Intro to Python for beginners, by Jessica McKellar (PyCon 2013)](https://youtu.be/rkx5_MRAV3A)
_Video_ - [Intro to Python for beginners, by Jessica McKellar (PyCon 2013)](https://youtu.be/rkx5_MRAV3A)
*Video* - [Breaking the rules, by Jessica McKellar (PyCon Sweden)](https://youtu.be/C0fnHhY9UOc)
_Video_ - [Breaking the rules, by Jessica McKellar (PyCon Sweden)](https://youtu.be/C0fnHhY9UOc)
*Video* - [Build & break a Python sandbox, by Jessica McKellar (PyCon 2014)](https://pyvideo.org/pycon-us-2014/building-and-breaking-a-python-sandbox.html)
_Video_ - [Build & break a Python sandbox, by Jessica McKellar (PyCon 2014)](https://pyvideo.org/pycon-us-2014/building-and-breaking-a-python-sandbox.html)
*Video* - [Cache me if you can, by Guillaume Ardaud (PyCon 2014)](https://pyvideo.org/pycon-us-2014/cache-me-if-you-can-memcached-caching-patterns.html)
_Video_ - [Cache me if you can, by Guillaume Ardaud (PyCon 2014)](https://pyvideo.org/pycon-us-2014/cache-me-if-you-can-memcached-caching-patterns.html)
*Video* - [Loop like a native, by Ned Batchelder (PyCon 2013)](https://youtu.be/EnSu9hHGq5o)
_Video_ - [Loop like a native, by Ned Batchelder (PyCon 2013)](https://youtu.be/EnSu9hHGq5o)
*Video* - [Modern Dictionaries, by Raymond Hettinger (SF Python)](https://youtu.be/p33CVV29OG8)
_Video_ - [Modern Dictionaries, by Raymond Hettinger (SF Python)](https://youtu.be/p33CVV29OG8)
*Video* - [Python Language, by Guido van Rossum (PyCon 2016)](https://youtu.be/YgtL4S7Hrwo)
_Video_ - [Python Language, by Guido van Rossum (PyCon 2016)](https://youtu.be/YgtL4S7Hrwo)
*Video* - [The Mighty Dictionary, by Brandon Rhodes (PyCon 2010)](https://pyvideo.org/pycon-us-2010/the-mighty-dictionary-55.html)
_Video_ - [The Mighty Dictionary, by Brandon Rhodes (PyCon 2010)](https://pyvideo.org/pycon-us-2010/the-mighty-dictionary-55.html)
*Article* - [Static types in Python, oh my(py)!](https://blog.zulip.org/2016/10/13/static-types-in-python-oh-mypy)
_Article_ - [Static types in Python, oh my(py)!](https://blog.zulip.org/2016/10/13/static-types-in-python-oh-mypy)
*Guide* - [The Hitchhiker’s Guide to Python!](https://docs.python-guide.org/)
_Guide_ - [The Hitchhiker’s Guide to Python!](https://docs.python-guide.org/)
## Java/Android
*Course* - [Android Development for Beginners](https://www.udacity.com/course/android-development-for-beginners--ud837)
_Course_ - [Android Development for Beginners](https://www.udacity.com/course/android-development-for-beginners--ud837)
*Blog* - [Java Tutorials for Beginners](https://www.geeksforgeeks.org/java/)
_Blog_ - [Java Tutorials for Beginners](https://www.geeksforgeeks.org/java/)
## JavaScript/ECMAScript
*Tutorial* - [clean-code-javascript Software engineering principles](https://github.com/ryanmcdermott/clean-code-javascript)
_Tutorial_ - [clean-code-javascript Software engineering principles](https://github.com/ryanmcdermott/clean-code-javascript)
*Course* - [React native and redux course](https://www.udemy.com/course/the-complete-react-native-and-redux-course/) (*Not free!*)
_Course_ - [React native and redux course](https://www.udemy.com/course/the-complete-react-native-and-redux-course/) (_Not free!_)
*Slides* - [TypeScript vs. CoffeeScript vs. ES6](https://www.slideshare.net/NeilGreen1/type-script-vs-coffeescript-vs-es6)
_Slides_ - [TypeScript vs. CoffeeScript vs. ES6](https://www.slideshare.net/NeilGreen1/type-script-vs-coffeescript-vs-es6)
## TypeScript
*Tutorial* - [TypeScript handbook section on base types](https://www.typescriptlang.org/docs/handbook/basic-types.html)
_Tutorial_ - [TypeScript handbook section on base types](https://www.typescriptlang.org/docs/handbook/basic-types.html)
*Book* - [TypeScript Deep Dive](https://basarat.gitbooks.io/typescript/)
_Book_ - [TypeScript Deep Dive](https://basarat.gitbooks.io/typescript/)
*Guide* - [TypeScript Declaration Files Introduction](https://www.typescriptlang.org/docs/handbook/declaration-files/introduction.html)
_Guide_ - [TypeScript Declaration Files Introduction](https://www.typescriptlang.org/docs/handbook/declaration-files/introduction.html)
## Git/version control systems (VCS)
@@ -94,31 +93,31 @@ You may want to take a look first at our [Git and GitHub guide](../git/index.md)
## Computer science/algorithms
*Blog* - [GeeksforGeeks](https://www.geeksforgeeks.org)
_Blog_ - [GeeksforGeeks](https://www.geeksforgeeks.org)
*Book* [Introduction to Algorithms](https://mitpress.mit.edu/books/introduction-algorithms) (*Not free!*)
_Book_ [Introduction to Algorithms](https://mitpress.mit.edu/books/introduction-algorithms) (_Not free!_)
*Blog* - [Setosa data visualization and visual explanations](https://setosa.io)
_Blog_ - [Setosa data visualization and visual explanations](https://setosa.io)
*Course* - [Algorithms, Part I](https://www.coursera.org/learn/algorithms-part1)
_Course_ - [Algorithms, Part I](https://www.coursera.org/learn/algorithms-part1)
*Course* - [Open Source Society University](https://ossu.firebaseapp.com)
_Course_ - [Open Source Society University](https://ossu.firebaseapp.com)
*Course* - [MIT CSAIL 6.828: Operative Systems Engineering](https://pdos.csail.mit.edu/6.828/2016)
_Course_ - [MIT CSAIL 6.828: Operative Systems Engineering](https://pdos.csail.mit.edu/6.828/2016)
## Community experience
*Book* - [Producing Open Source Software](https://producingoss.com/en/)
_Book_ - [Producing Open Source Software](https://producingoss.com/en/)
*Article* - [Advice on Starting And Running A New Open Source Project](https://www.harihareswara.net/sumana/2016/08/04/1)
_Article_ - [Advice on Starting And Running A New Open Source Project](https://www.harihareswara.net/sumana/2016/08/04/1)
*Article* - [How to ask good questions](https://jvns.ca/blog/good-questions)
_Article_ - [How to ask good questions](https://jvns.ca/blog/good-questions)
*Article* - [Notes for New FLOSS Contributors](https://www.harihareswara.net/sumana/2016/10/12/0)
_Article_ - [Notes for New FLOSS Contributors](https://www.harihareswara.net/sumana/2016/10/12/0)
*Article* - [To be mentored](https://trueskawka.github.io/zulip/outreachy/blog/2017/01/02/to-be-mentored.html)
_Article_ - [To be mentored](https://trueskawka.github.io/zulip/outreachy/blog/2017/01/02/to-be-mentored.html)
*Article* - [To mentor](https://trueskawka.github.io/zulip/outreachy/gci/blog/2017/01/03/to-mentor.html)
_Article_ - [To mentor](https://trueskawka.github.io/zulip/outreachy/gci/blog/2017/01/03/to-mentor.html)
[List of good projects for new contributors](https://github.com/MunGell/awesome-for-beginners)

16
docs/tutorials/screenshot-and-gif-software.md
View File

@@ -6,13 +6,16 @@ requests, as other contributors can see the changes you have made without having
to check out your branch.
## Screenshot tools by platform
### Browser
- Firefox can take screenshots without any plugins (stable feature starting from v57)
- You can find it under `page actions / Take a screenshot`
- It is capable of screenshotting the entire page, visible area and individual DOM elements
- You can find it under `page actions / Take a screenshot`
- It is capable of screenshotting the entire page, visible area and individual DOM elements
- [LightShot Screenshot](https://app.prntscr.com/en/index.html) (Chrome, Firefox, IE & Opera)
### macOS
- Command-Shift-3 to capture the entire screen
- Command-Shift-4 and drag cursor to select a specific area
- Command-Shift-5 and select suitable option to capture or record screen (macOS Mojave)
@@ -20,6 +23,7 @@ to check out your branch.
- [Gyazo](https://gyazo.com/en)
### Windows
- PrtScn to copy the screen to the clipboard
- Windows-PrtScn to save the screen to a file
- Snipping Tool (inbuilt)
@@ -27,15 +31,19 @@ to check out your branch.
- [Gyazo](https://gyazo.com/en)
### Linux
- gnome-screenshot (inbuilt, you can use Ctrl-Shift-PrtScn as a shortcut for its “select an area to grab” feature)
## GIF tools by platform
### Browser
- [GIPHY](https://giphy.com)
- [Chrome Capture](https://chrome.google.com/webstore/detail/chrome-capture/ggaabchcecdbomdcnbahdfddfikjmphe?hl=en)
(Tip: Use Alt+i to interact with the page while recording)
(Tip: Use Alt+i to interact with the page while recording)
### macOS
- [QuickTime](https://support.apple.com/en-in/HT201066)
- [GIPHY](https://giphy.com/apps/giphycapture)
- [CloudApp](https://www.getcloudapp.com)
@@ -45,10 +53,12 @@ to check out your branch.
- [Gyazo GIF](https://gyazo.com/en)
### Windows
- [ScreenToGif](https://www.screentogif.com)
- [Gyazo GIF](https://gyazo.com/en)
- [Monosnap](https://www.monosnap.com/welcome)
### Linux
- [Peek](https://github.com/phw/peek)
- [SilentCast](https://github.com/colinkeenan/silentcast)

20
docs/tutorials/shell-tips.md
View File

@@ -1,7 +1,7 @@
# Shell tips
The *shell* is a **command line interpreter**. To use it you can open a
*terminal* (sometimes called a *console*). This is how most terminal windows
The _shell_ is a **command line interpreter**. To use it you can open a
_terminal_ (sometimes called a _console_). This is how most terminal windows
look like:
![An example shell window](../images/shell-screenshot.png)
@@ -26,12 +26,12 @@ awaiting new orders. The prompt can contain useful information, let's look
at `(venv)john@laptop:~$`:
- `(venv)` informs the user that they're currently in a virtual environment
(more on [Python virtual
(more on [Python virtual
environments](https://docs.python-guide.org/dev/virtualenvs/))
- the `john` before `@` is the username
- the `laptop` is the host machine name
- the `~` after the colon informs the user they're currently in the home
folder of the user `john`
folder of the user `john`
You shouldn't type the prompt or the text preceding it, since it isn't a
part of the commands.
@@ -89,7 +89,7 @@ shell that the following command must be run as the root - a user that by
default has access to all commands and files on a Unix operating system (i.e.
a user with administrator privileges). That's why you may be asked for a
password in those cases: the system verifies you have permission to act as
the *root* user.
the _root_ user.
In case you were wondering, the name `sudo` comes from **s**uper **u**ser
**do**.
@@ -211,9 +211,9 @@ $ cd /home/john/notes
Here, the command is `cd`, and the first (and only) argument is
`/home/john/notes`:
- `cd` - *command*: changes your current directory.
- `cd` - _command_: changes your current directory.
- `/home/john/notes` - *argument*: the directory where you want to go.
- `/home/john/notes` - _argument_: the directory where you want to go.
In each command the arguments are specified in different ways, and have
different meanings.
@@ -232,13 +232,13 @@ In this case, we're saying: "Bash, use the app `nano` to open the file
`todo.txt`, enabling mouse support, and saving the backup files to
`/home/john/backups`". The different parts are:
- `nano` - *command*: program that allows editing text easily.
- `nano` - _command_: program that allows editing text easily.
- `-C` - *argument*: needs you to indicate where the backups should be stored,
- `-C` - _argument_: needs you to indicate where the backups should be stored,
and thus you have to add an additional argument after it, to specify the
directory (`/home/john/backups` in the example).
- `--mouse` - *argument*: is just an option you set, `nano` doesn't need
- `--mouse` - _argument_: is just an option you set, `nano` doesn't need
anything else to make it work. Thus, there isn't any extra argument for that.
Note that the `todo.txt` is the file we want to open! It has nothing to do with

10
docs/tutorials/writing-views.md
View File

@@ -100,7 +100,6 @@ def home(request: HttpRequest) -> HttpResponse:
Templates for the main website are found in
[templates/zerver/app](https://github.com/zulip/zulip/tree/main/templates/zerver/app).
## Writing API REST endpoints
These are code-parseable views that take x-www-form-urlencoded or JSON
@@ -114,7 +113,7 @@ This method will authenticate the user either through a session token
from a cookie on the browser, or from a base64 encoded `email:api-key`
string given via HTTP basic auth for API clients.
``` py
```py
>>> import requests
>>> r = requests.get('https://api.github.com/user', auth=('hello@example.com', '0123456789abcdeFGHIJKLmnopQRSTUV'))
>>> r.status_code
@@ -146,7 +145,7 @@ arguments for.
Here's an example:
``` py
```py
from zerver.decorator import require_realm_admin
from zerver.lib.request import has_request_variables, REQ
@@ -232,7 +231,7 @@ If you're removing data, use DELETE.
When writing a new API endpoint, with the exception of things like
sending messages, requests should be safe to repeat, without impacting
the state of the server. This is *idempotency*.
the state of the server. This is _idempotency_.
You will often want to return an error if a request to change
something would do nothing because the state is already as desired, to
@@ -343,7 +342,7 @@ If the webhook does not have an option to provide a bot email, use the
`webhook_view` decorator, to fill in the `user_profile` and
`request.client` fields of a request:
``` py
```py
@webhook_view('PagerDuty')
@has_request_variables
def api_pagerduty_webhook(request, user_profile,
@@ -351,6 +350,7 @@ def api_pagerduty_webhook(request, user_profile,
stream=REQ(default='pagerduty'),
topic=REQ(default=None)):
```
`request.client` will be the result of `get_client("ZulipPagerDutyWebhook")`
in this example and it will be passed to `check_send_stream_message`. For
more information, see [Clients in Zulip](../subsystems/client.md).