From d81ce3ba76a9dcdf9ff7587ec7c44ef0d3bf8d2e Mon Sep 17 00:00:00 2001 From: Anders Kaseorg Date: Fri, 20 Aug 2021 13:54:08 -0700 Subject: [PATCH] docs: Format Markdown with Prettier. Signed-off-by: Anders Kaseorg (cherry picked from commit a4dbc1edd4d167549552d10e8a780d06b2cd3602) --- .github/pull_request_template.md | 3 - CONTRIBUTING.md | 11 +- docs/contributing/accessibility.md | 1 - docs/contributing/bug-reports.md | 7 +- docs/contributing/code-reviewing.md | 35 +- docs/contributing/code-style.md | 88 ++-- docs/contributing/gsoc-ideas.md | 46 +- docs/contributing/summer-with-zulip.md | 2 - docs/contributing/version-control.md | 134 +++--- docs/contributing/zulipbot-usage.md | 42 +- docs/development/authentication.md | 53 ++- docs/development/remote.md | 67 +-- docs/development/setup-advanced.md | 3 +- docs/development/setup-vagrant.md | 23 +- docs/development/test-install.md | 15 +- docs/development/using.md | 11 +- docs/documentation/api.md | 7 +- docs/documentation/integrations.md | 23 +- docs/documentation/openapi.md | 1 + docs/documentation/overview.md | 20 +- docs/documentation/user.md | 47 +- docs/git/cheat-sheet.md | 138 +++--- docs/git/collaborate.md | 10 +- docs/git/fixing-commits.md | 10 + docs/git/overview.md | 12 +- docs/git/pull-requests.md | 6 +- docs/git/terminology.md | 15 +- docs/git/the-git-difference.md | 18 +- docs/git/troubleshooting.md | 6 +- docs/git/using.md | 5 +- docs/git/zulip-tools.md | 5 +- docs/overview/architecture-overview.md | 98 ++-- docs/overview/changelog.md | 88 ++-- docs/overview/directory-structure.md | 35 +- docs/overview/release-lifecycle.md | 14 +- docs/production/authentication-methods.md | 193 ++++---- docs/production/deployment.md | 194 ++++---- docs/production/email-gateway.md | 58 +-- docs/production/email.md | 1 + docs/production/expensive-migrations.md | 6 +- docs/production/export-and-import.md | 70 +-- docs/production/giphy-gif-integration.md | 2 - docs/production/install-existing-server.md | 2 +- docs/production/install.md | 37 +- docs/production/maintain-secure-upgrade.md | 1 + docs/production/mobile-push-notifications.md | 69 +-- docs/production/password-strength.md | 8 +- docs/production/postgresql.md | 4 +- docs/production/requirements.md | 30 +- docs/production/security-model.md | 13 +- docs/production/settings.md | 2 + docs/production/ssl-certificates.md | 13 +- docs/production/troubleshooting.md | 4 + docs/production/upgrade-or-modify.md | 241 +++++----- docs/production/upload-backends.md | 26 +- docs/subsystems/analytics.md | 6 +- docs/subsystems/api-release-checklist.md | 2 +- docs/subsystems/billing.md | 2 + docs/subsystems/caching.md | 7 +- docs/subsystems/dependencies.md | 11 +- docs/subsystems/email.md | 1 + docs/subsystems/emoji.md | 1 + docs/subsystems/full-text-search.md | 10 +- docs/subsystems/hashchange-system.md | 4 +- docs/subsystems/hotspots.md | 6 +- docs/subsystems/html-css.md | 31 +- docs/subsystems/input-pills.md | 1 - docs/subsystems/logging.md | 8 +- docs/subsystems/management-commands.md | 2 +- docs/subsystems/markdown.md | 1 - docs/subsystems/notifications.md | 59 +-- docs/subsystems/performance.md | 5 +- docs/subsystems/pointer.md | 2 +- docs/subsystems/realms.md | 6 +- docs/subsystems/release-checklist.md | 1 + docs/subsystems/schema-migrations.md | 12 +- docs/subsystems/sending-messages.md | 171 +++---- docs/subsystems/settings.md | 7 +- docs/subsystems/widgets.md | 15 +- docs/testing/continuous-integration.md | 32 +- docs/testing/linters.md | 2 + docs/testing/manual-testing.md | 446 +++++++++--------- docs/testing/philosophy.md | 15 +- docs/testing/testing-with-django.md | 59 +-- docs/testing/testing-with-node.md | 50 +- docs/testing/testing-with-puppeteer.md | 1 + docs/testing/testing.md | 12 +- docs/testing/typescript.md | 3 +- docs/translating/chinese.md | 28 +- docs/translating/french.md | 4 +- docs/translating/german.md | 140 +++--- docs/translating/hindi.md | 19 +- docs/translating/internationalization.md | 14 +- docs/translating/polish.md | 69 ++- docs/translating/spanish.md | 29 +- docs/translating/translating.md | 35 +- docs/tutorials/life-of-a-request.md | 2 +- docs/tutorials/new-feature-tutorial.md | 71 +-- docs/tutorials/reading-list.md | 93 ++-- docs/tutorials/screenshot-and-gif-software.md | 16 +- docs/tutorials/shell-tips.md | 20 +- docs/tutorials/writing-views.md | 10 +- static/shared/README.md | 26 +- tools/droplets/README.md | 4 +- tools/oneclickapps/README.md | 7 +- zilencer/README.md | 3 +- 106 files changed, 1912 insertions(+), 1742 deletions(-) diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index a541423270..793082dfe5 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -1,14 +1,11 @@ - **Testing plan:** - **GIFs or screenshots:** - diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 79b0658e57..541ef793fc 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -26,6 +26,7 @@ To make a code or documentation contribution, read our [step-by-step guide](#your-first-codebase-contribution) to getting started with the Zulip codebase. A small sample of the type of work that needs doing: + - Bug squashing and feature development on our Python/Django [backend](https://github.com/zulip/zulip), web [frontend](https://github.com/zulip/zulip), React Native @@ -57,6 +58,7 @@ This section has a step by step guide to starting as a Zulip codebase contributor. It's long, but don't worry about doing all the steps perfectly; no one gets it right the first time, and there are a lot of people available to help. + - First, make an account on the [Zulip community server](https://zulip.readthedocs.io/en/latest/contributing/chat-zulip-org.html), paying special attention to the community norms. If you'd like, introduce @@ -119,6 +121,7 @@ have a new feature you'd like to add, we recommend you start by posting in feature idea and the problem that you're hoping to solve. Other notes: + - For a first pull request, it's better to aim for a smaller contribution than a bigger one. Many first contributions have fewer than 10 lines of changes (not counting changes to tests). @@ -215,9 +218,9 @@ and how to reproduce it if known, your browser/OS if relevant, and a if appropriate. **Reporting security issues**. Please do not report security issues - publicly, including on public streams on chat.zulip.org. You can - email security@zulip.com. We create a CVE for every security - issue in our released software. +publicly, including on public streams on chat.zulip.org. You can +email security@zulip.com. We create a CVE for every security +issue in our released software. ## User feedback @@ -252,6 +255,7 @@ summer interns from Harvard, MIT, and Stanford. While each third-party program has its own rules and requirements, the Zulip community's approaches all of these programs with these ideas in mind: + - We try to make the application process as valuable for the applicant as possible. Expect high-quality code reviews, a supportive community, and publicly viewable patches you can link to from your resume, regardless of @@ -307,6 +311,7 @@ for ZSoC, we'll contact you when the GSoC results are announced. perception of projects like Zulip. We've collected a few sites below where we know Zulip has been discussed. Doing everything in the following list typically takes about 15 minutes. + - Star us on GitHub. There are four main repositories: [server/web](https://github.com/zulip/zulip), [mobile](https://github.com/zulip/zulip-mobile), diff --git a/docs/contributing/accessibility.md b/docs/contributing/accessibility.md index 9886250583..b6cda6f1a2 100644 --- a/docs/contributing/accessibility.md +++ b/docs/contributing/accessibility.md @@ -79,7 +79,6 @@ possible, the following resources may be useful. - The [MDN page on accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility) - The [Open edX Accessibility Guidelines][openedx-guidelines] for developers - [chrome-webstore]: https://chrome.google.com/webstore/detail/accessibility-developer-t/fpkknkljclfencbdbgkenhalefipecmb [openedx-guidelines]: https://edx.readthedocs.io/projects/edx-developer-guide/en/latest/conventions/accessibility.html [accessibility-issues]: https://github.com/zulip/zulip/issues?q=is%3Aissue+is%3Aopen+label%3A%22area%3A%20accessibility%22 diff --git a/docs/contributing/bug-reports.md b/docs/contributing/bug-reports.md index 9ec9f808eb..b7d7aef103 100644 --- a/docs/contributing/bug-reports.md +++ b/docs/contributing/bug-reports.md @@ -9,14 +9,15 @@ Please include these elements in your bug report to make it easier for us to hel - Steps to take in order to reproduce the buggy behavior - Whether you are using Zulip in production or in the development -environment, and whether these are old versions + environment, and whether these are old versions - Whether you are using the web app, a desktop app or a mobile device -to access Zulip + to access Zulip - Any additional information that would help: screenshots, GIFs, a -pastebin of the error log + pastebin of the error log Further reading: + - [How to write a bug report that will make your engineers love you](https://testlio.com/blog/the-ideal-bug-report/) - [How to Report Bugs Effectively](https://www.chiark.greenend.org.uk/~sgtatham/bugs.html) diff --git a/docs/contributing/code-reviewing.md b/docs/contributing/code-reviewing.md index 9e85f7012c..cdd75b5391 100644 --- a/docs/contributing/code-reviewing.md +++ b/docs/contributing/code-reviewing.md @@ -10,6 +10,7 @@ When you send a PR, try to think of a good person to review it -- outside of the handful of people who do a ton of reviews -- and `@`-mention them with something like "`@person`, would you review this?". Good choices include + - someone based in your timezone or a nearby timezone - people working on similar things, or in a loosely related area @@ -106,7 +107,7 @@ sooner is better. ## Things to look for -- *The CI build.* The tests need to pass. One can investigate +- _The CI build._ The tests need to pass. One can investigate any failures and figure out what to fix by clicking on a red X next to the commit hash or the Detail links on a pull request. (Example: in [#17584](https://github.com/zulip/zulip/pull/17584), @@ -120,25 +121,25 @@ sooner is better. See our docs on [continuous integration](../testing/continuous-integration.md) to learn more. -- *Technical design.* There are a lot of considerations here: +- _Technical design._ There are a lot of considerations here: security, migration paths/backwards compatibility, cost of new dependencies, interactions with features, speed of performance, API changes. Security is especially important and worth thinking about carefully with any changes to security-sensitive code like views. -- *User interface and visual design.* If frontend changes are +- _User interface and visual design._ If frontend changes are involved, the reviewer will check out the code, play with the new UI, and verify it for both quality and consistency with the rest of the Zulip UI. We highly encourage posting screenshots to save reviewers time in getting a feel for what the feature looks like -- you'll get a quicker response that way. -- *Error handling.* The code should always check for invalid user +- _Error handling._ The code should always check for invalid user input. User-facing error messages should be clear and when possible be actionable (it should be obvious to the user what they need to do in order to correct the problem). -- *Testing.* The tests should validate that the feature works +- _Testing._ The tests should validate that the feature works correctly, and specifically test for common error conditions, bad user input, and potential bugs that are likely for the type of change being made. Tests that exclude whole classes of potential @@ -147,16 +148,16 @@ sooner is better. Markdown processors](../subsystems/markdown.md), or the `GetEventsTest` test for buggy race condition handling). -- *Translation.* Make sure that the strings are marked for +- _Translation._ Make sure that the strings are marked for [translation]. -- *Clear function, argument, variable, and test names.* Every new +- _Clear function, argument, variable, and test names._ Every new piece of Zulip code will be read many times by other developers, and future developers will grep for relevant terms when researching a problem, so it's important that variable names communicate clearly the purpose of each piece of the codebase. -- *Duplicated code.* Code duplication is a huge source of bugs in +- _Duplicated code._ Code duplication is a huge source of bugs in large projects and makes the codebase difficult to understand, so we avoid significant code duplication wherever possible. Sometimes avoiding code duplication involves some refactoring of existing @@ -164,32 +165,32 @@ sooner is better. commits (not squashed into other changes or left as a thing to do later). That series of commits can be in the same pull request as the feature that they support, and we recommend ordering the history - of commits so that the refactoring comes *before* the feature. That + of commits so that the refactoring comes _before_ the feature. That way, it's easy to merge the refactoring (and minimize risk of merge conflicts) if there are still user experience issues under discussion for the feature itself. -- *Completeness.* For refactorings, verify that the changes are +- _Completeness._ For refactorings, verify that the changes are complete. Usually one can check that efficiently using `git grep`, and it's worth it, as we very frequently find issues by doing so. -- *Documentation updates.* If this changes how something works, does it +- _Documentation updates._ If this changes how something works, does it update the documentation in a corresponding way? If it's a new feature, is it documented, and documented in the right place? -- *Good comments.* It's often worth thinking about whether explanation +- _Good comments._ It's often worth thinking about whether explanation in a commit message or pull request discussion should be included in a comment, `/docs`, or other documentation. But it's better yet if verbose explanation isn't needed. We prefer writing code that is readable without explanation over a heavily commented codebase using lots of clever tricks. -- *Coding style.* See the Zulip [code-style] documentation for +- _Coding style._ See the Zulip [code-style] documentation for details. Our goal is to have as much of this as possible verified via the linters and tests, but there's always going to be unusual forms of Python/JavaScript style that our tools don't check for. -- *Clear commit messages.* See the [Zulip version +- _Clear commit messages._ See the [Zulip version control][commit-messages] documentation for details on what we look for. @@ -197,15 +198,15 @@ sooner is better. Some points specific to the Zulip server codebase: -- *Testing -- Backend.* We are trying to maintain ~100% test coverage +- _Testing -- Backend._ We are trying to maintain ~100% test coverage on the backend, so backend changes should have negative tests for the various error conditions. -- *Testing -- Frontend.* If the feature involves frontend changes, +- _Testing -- Frontend._ If the feature involves frontend changes, there should be frontend tests. See the [test writing][test-writing] documentation for more details. -- *mypy annotations.* New functions should be annotated using [mypy] +- _mypy annotations._ New functions should be annotated using [mypy] and existing annotations should be updated. Use of `Any`, `ignore`, and unparameterized containers should be limited to cases where a more precise type cannot be specified. diff --git a/docs/contributing/code-style.md b/docs/contributing/code-style.md index 48aa63fe4d..f40a74b739 100644 --- a/docs/contributing/code-style.md +++ b/docs/contributing/code-style.md @@ -48,15 +48,15 @@ The Vagrant setup process runs this for you. `lint` runs many lint checks in parallel, including -- JavaScript ([ESLint](https://eslint.org/), - [Prettier](https://prettier.io/)) -- Python ([mypy](http://mypy-lang.org/), - [Pyflakes](https://pypi.python.org/pypi/pyflakes), - [Black](https://github.com/psf/black), - [isort](https://pycqa.github.io/isort/)) -- templates -- Puppet configuration -- custom checks (e.g. trailing whitespace and spaces-not-tabs) +- JavaScript ([ESLint](https://eslint.org/), + [Prettier](https://prettier.io/)) +- Python ([mypy](http://mypy-lang.org/), + [Pyflakes](https://pypi.python.org/pypi/pyflakes), + [Black](https://github.com/psf/black), + [isort](https://pycqa.github.io/isort/)) +- templates +- Puppet configuration +- custom checks (e.g. trailing whitespace and spaces-not-tabs) ## Secrets @@ -143,9 +143,9 @@ some_objs = UserProfile.objects.get(id=17) assert obj.id in set([o.id for i in some_objs]) ``` -### user\_profile.save() +### user_profile.save() -You should always pass the update\_fields keyword argument to .save() +You should always pass the update_fields keyword argument to .save() when modifying an existing Django model object. By default, .save() will overwrite every value in the column, which results in lots of race conditions where unrelated changes made by one thread can be @@ -155,7 +155,7 @@ object before the first thread wrote out its change. ### Using raw saves to update important model objects In most cases, we already have a function in zerver/lib/actions.py with -a name like do\_activate\_user that will correctly handle lookups, +a name like do_activate_user that will correctly handle lookups, caching, and notifying running browsers via the event system about your change. So please check whether such a function exists before writing new code to modify a model object, since your new code has a good chance @@ -172,6 +172,7 @@ libraries as `from datetime import datetime, timezone` and `from django.utils.timezone import now as timezone_now`. Use: + - `timezone_now()` to get a datetime when Django is available, such as in `zerver/`. - `datetime.now(tz=timezone.utc)` when Django is not available, such as @@ -189,6 +190,7 @@ parameter, `datetime.utcnow()` and `datetime.utcfromtimestamp()`, and the end. Additional notes: + - Especially in scripts and puppet configuration where Django is not available, using `time.time()` to get timestamps can be cleaner than dealing with datetimes. @@ -295,11 +297,11 @@ We used to favor attaching behaviors in templates like so: but there are some reasons to prefer attaching events using jQuery code: -- Potential huge performance gains by using delegated events where - possible -- When calling a function from an `onclick` attribute, `this` is not - bound to the element like you might think -- jQuery does event normalization +- Potential huge performance gains by using delegated events where + possible +- When calling a function from an `onclick` attribute, `this` is not + bound to the element like you might think +- jQuery does event normalization Either way, avoid complicated JavaScript code inside HTML attributes; call a helper function instead. @@ -322,34 +324,34 @@ type changes in the future. ### Python -- Our Python code is formatted with - [Black](https://github.com/psf/black) and - [isort](https://pycqa.github.io/isort/). The [linter - tool](../testing/linters.md) enforces this by running Black and - isort in check mode, or in write mode with - `tools/lint --only=black,isort --fix`. You may find it helpful to - [integrate - Black](https://black.readthedocs.io/en/stable/editor_integration.html) - and - [isort](https://pycqa.github.io/isort/#installing-isorts-for-your-preferred-text-editor) - with your editor. -- Don't put a shebang line on a Python file unless it's meaningful to - run it as a script. (Some libraries can also be run as scripts, e.g. - to run a test suite.) -- Scripts should be executed directly (`./script.py`), so that the - interpreter is implicitly found from the shebang line, rather than - explicitly overridden (`python script.py`). -- Put all imports together at the top of the file, absent a compelling - reason to do otherwise. -- Unpacking sequences doesn't require list brackets: +- Our Python code is formatted with + [Black](https://github.com/psf/black) and + [isort](https://pycqa.github.io/isort/). The [linter + tool](../testing/linters.md) enforces this by running Black and + isort in check mode, or in write mode with + `tools/lint --only=black,isort --fix`. You may find it helpful to + [integrate + Black](https://black.readthedocs.io/en/stable/editor_integration.html) + and + [isort](https://pycqa.github.io/isort/#installing-isorts-for-your-preferred-text-editor) + with your editor. +- Don't put a shebang line on a Python file unless it's meaningful to + run it as a script. (Some libraries can also be run as scripts, e.g. + to run a test suite.) +- Scripts should be executed directly (`./script.py`), so that the + interpreter is implicitly found from the shebang line, rather than + explicitly overridden (`python script.py`). +- Put all imports together at the top of the file, absent a compelling + reason to do otherwise. +- Unpacking sequences doesn't require list brackets: - ```python - [x, y] = xs # unnecessary - x, y = xs # better - ``` + ```python + [x, y] = xs # unnecessary + x, y = xs # better + ``` -- For string formatting, use `x % (y,)` rather than `x % y`, to avoid - ambiguity if `y` happens to be a tuple. +- For string formatting, use `x % (y,)` rather than `x % y`, to avoid + ambiguity if `y` happens to be a tuple. ### Tests diff --git a/docs/contributing/gsoc-ideas.md b/docs/contributing/gsoc-ideas.md index 87891c671b..993ab402f1 100644 --- a/docs/contributing/gsoc-ideas.md +++ b/docs/contributing/gsoc-ideas.md @@ -138,19 +138,19 @@ Also, you're going to find that people give you links to pages that answer your questions. Here's how that often works: 1. you [try to solve your problem until you get stuck, including -looking through our code and our documentation, then start formulating -your request for -help](https://blogs.akamai.com/2013/10/you-must-try-and-then-you-must-ask.html) + looking through our code and our documentation, then start formulating + your request for + help](https://blogs.akamai.com/2013/10/you-must-try-and-then-you-must-ask.html) 1. you ask your question 1. someone directs you to a document 1. you go read that document, and try to use it to answer your question 1. you find you are confused about a new thing 1. you ask another question 1. now that you have demonstrated that you have the ability to read, -think, and learn new things, someone has a longer talk with you to -answer your new specific question + think, and learn new things, someone has a longer talk with you to + answer your new specific question 1. you and the other person collaborate to improve the document that you -read in step 3 :-) + read in step 3 :-) This helps us make a balance between person-to-person discussion and documentation that everyone can read, so we save time answering common @@ -484,17 +484,17 @@ CSS](https://github.com/zulip/zulip/). language and API design. Expert: Depends on language. - Develop [**@zulipbot**](https://github.com/zulip/zulipbot), the GitHub -workflow bot for the Zulip organization and its repositories. By utilizing the -[GitHub API](https://developer.github.com/v3/), -[**@zulipbot**](https://github.com/zulipbot) improves the experience of Zulip -contributors by managing the issues and pull requests in the Zulip repositories, -such as assigning issues to contributors and appropriately labeling issues with -their current status to help contributors gain a better understanding of which -issues are being worked on. Since the project is in its early stages of -development, there are a variety of possible tasks that can be done, including -adding new features, writing unit tests and creating a testing framework, and -writing documentation. **Skills required**: Node.js, ECMAScript 6, and API -experience. Experts: Cynthia Lin, Joshua Pan. + workflow bot for the Zulip organization and its repositories. By utilizing the + [GitHub API](https://developer.github.com/v3/), + [**@zulipbot**](https://github.com/zulipbot) improves the experience of Zulip + contributors by managing the issues and pull requests in the Zulip repositories, + such as assigning issues to contributors and appropriately labeling issues with + their current status to help contributors gain a better understanding of which + issues are being worked on. Since the project is in its early stages of + development, there are a variety of possible tasks that can be done, including + adding new features, writing unit tests and creating a testing framework, and + writing documentation. **Skills required**: Node.js, ECMAScript 6, and API + experience. Experts: Cynthia Lin, Joshua Pan. ### React Native mobile app @@ -522,11 +522,11 @@ summer. We'd love to have multiple students working on this area if we have enough strong applicants. **Skills required**: Strong programming experience, especially in - reading the documentation of unfamiliar projects and communicating - what you learned. JavaScript and React experience are great pluses, - as are iOS or Android development/design experience is useful as - well. You'll need to learn React Native as part of getting - involved. There's tons of good online tutorials, courses, etc. +reading the documentation of unfamiliar projects and communicating +what you learned. JavaScript and React experience are great pluses, +as are iOS or Android development/design experience is useful as +well. You'll need to learn React Native as part of getting +involved. There's tons of good online tutorials, courses, etc. ### Electron desktop app @@ -542,7 +542,7 @@ Experts: Anders Kaseorg, Akash Nimare, Abhighyan Khaund. the open issues and get involved! **Skills required**: JavaScript experience, Electron experience. You - can learn electron as part of your application! +can learn electron as part of your application! Good preparation for desktop app projects is to (1) try out the app and see if you can find bugs or polish problems lacking open issues diff --git a/docs/contributing/summer-with-zulip.md b/docs/contributing/summer-with-zulip.md index ff0e592830..492378d524 100644 --- a/docs/contributing/summer-with-zulip.md +++ b/docs/contributing/summer-with-zulip.md @@ -225,7 +225,6 @@ materials](https://developers.google.com/open-source/gsoc/resources/manual). your contributions to the open source world this summer will be something you can be proud of for the rest of your life. - ## What makes a successful summer Success for the student means a few things, in order of importance: @@ -247,7 +246,6 @@ Success for the student means a few things, in order of importance: student has implemented. That section of code should be more readable, better-tested, and have clearer documentation. - ## Extra notes for mentors - You're personally accountable for your student having a successful summer. If diff --git a/docs/contributing/version-control.md b/docs/contributing/version-control.md index b37198016c..02feec9f64 100644 --- a/docs/contributing/version-control.md +++ b/docs/contributing/version-control.md @@ -11,51 +11,51 @@ helps a lot in preventing bugs. Commits must be coherent: -- It should pass tests (so test updates needed by a change should be - in the same commit as the original change, not a separate "fix the - tests that were broken by the last commit" commit). -- It should be safe to deploy individually, or explain in detail in - the commit message as to why it isn't (maybe with a [manual] tag). - So implementing a new API endpoint in one commit and then adding the - security checks in a future commit should be avoided -- the security - checks should be there from the beginning. -- Error handling should generally be included along with the code that - might trigger the error. -- TODO comments should be in the commit that introduces the issue or - the functionality with further work required. +- It should pass tests (so test updates needed by a change should be + in the same commit as the original change, not a separate "fix the + tests that were broken by the last commit" commit). +- It should be safe to deploy individually, or explain in detail in + the commit message as to why it isn't (maybe with a [manual] tag). + So implementing a new API endpoint in one commit and then adding the + security checks in a future commit should be avoided -- the security + checks should be there from the beginning. +- Error handling should generally be included along with the code that + might trigger the error. +- TODO comments should be in the commit that introduces the issue or + the functionality with further work required. Commits should generally be minimal: -- Significant refactorings should be done in a separate commit from - functional changes. -- Moving code from one file to another should be done in a separate - commits from functional changes or even refactoring within a file. -- 2 different refactorings should be done in different commits. -- 2 different features should be done in different commits. -- If you find yourself writing a commit message that reads like a list - of somewhat dissimilar things that you did, you probably should have - just done multiple commits. +- Significant refactorings should be done in a separate commit from + functional changes. +- Moving code from one file to another should be done in a separate + commits from functional changes or even refactoring within a file. +- 2 different refactorings should be done in different commits. +- 2 different features should be done in different commits. +- If you find yourself writing a commit message that reads like a list + of somewhat dissimilar things that you did, you probably should have + just done multiple commits. When not to be overly minimal: -- For completely new features, you don't necessarily need to split out - new commits for each little subfeature of the new feature. E.g., if - you're writing a new tool from scratch, it's fine to have the - initial tool have plenty of options/features without doing separate - commits for each one. That said, reviewing a 2000-line giant blob of - new code isn't fun, so please be thoughtful about submitting things - in reviewable units. -- Don't bother to split backend commits from frontend commits, even - though the backend can often be coherent on its own. +- For completely new features, you don't necessarily need to split out + new commits for each little subfeature of the new feature. E.g., if + you're writing a new tool from scratch, it's fine to have the + initial tool have plenty of options/features without doing separate + commits for each one. That said, reviewing a 2000-line giant blob of + new code isn't fun, so please be thoughtful about submitting things + in reviewable units. +- Don't bother to split backend commits from frontend commits, even + though the backend can often be coherent on its own. Other considerations: -- Overly fine commits are easy to squash later, but not vice versa. - So err toward small commits, and the code reviewer can advise on - squashing. -- If a commit you write doesn't pass tests, you should usually fix - that by amending the commit to fix the bug, not writing a new "fix - tests" commit on top of it. +- Overly fine commits are easy to squash later, but not vice versa. + So err toward small commits, and the code reviewer can advise on + squashing. +- If a commit you write doesn't pass tests, you should usually fix + that by amending the commit to fix the bug, not writing a new "fix + tests" commit on top of it. Zulip expects you to structure the commits in your pull requests to form a clean history before we will merge them. It's best to write your @@ -91,6 +91,7 @@ First, check out of commits with good commit messages. The first line of the commit message is the **summary**. The summary: + - is written in the imperative (e.g., "Fix ...", "Add ...") - is kept short (max 76 characters, ideally less), while concisely explaining what the commit does @@ -105,7 +106,7 @@ prefix "provision:", using lowercase "**p**". Next, "Improve performance of install npm." starts with a capital "**I**", uses imperative tense, and ends with a period. -> *provision: Improve performance of installing npm.* +> _provision: Improve performance of installing npm._ Here are some more positive examples: @@ -121,16 +122,15 @@ Here are some more positive examples: > gather_subscriptions: Fix exception handling bad input. +Compare "_gather_subscriptions: Fix exception handling bad input._" with: -Compare "*gather_subscriptions: Fix exception handling bad input.*" with: - -- "*gather_subscriptions was broken*", which doesn't explain how +- "_gather_subscriptions was broken_", which doesn't explain how it was broken (and isn't in the imperative) -- "*Fix exception when given bad input*", in which it's impossible to +- "_Fix exception when given bad input_", in which it's impossible to tell from the summary what part of the codebase was changed -- "*gather_subscriptions: Fixing exception when given bad input.*", +- "_gather_subscriptions: Fixing exception when given bad input._", not in the imperative -- "*gather_subscriptions: Fixed exception when given bad input.*", +- "_gather_subscriptions: Fixed exception when given bad input._", not in the imperative The summary is followed by a blank line, and then the body of the @@ -143,29 +143,29 @@ automatically catch common mistakes in the commit message itself. ### Message body: -- The body is written in prose, with full paragraphs; each paragraph should - be separated from the next by a single blank line. -- The body explains: - - why and how the change was made - - any manual testing you did in addition to running the automated tests - - any aspects of the commit that you think are questionable and - you'd like special attention applied to. -- If the commit makes performance improvements, you should generally - include some rough benchmarks showing that it actually improves the - performance. -- When you fix a GitHub issue, [mark that you've fixed the issue in - your commit - message](https://help.github.com/en/articles/closing-issues-via-commit-messages) - so that the issue is automatically closed when your code is merged. - Zulip's preferred style for this is to have the final paragraph of - the commit message read e.g. "Fixes: \#123.". -- Avoid `Partially fixes #1234`; GitHub's regular expressions ignore - the "partially" and close the issue. `Fixes part of #1234` is a good alternative. -- Any paragraph content in the commit message should be line-wrapped - to about 68 characters per line, but no more than 70, so that your - commit message will be reasonably readable in `git log` in a normal - terminal. You may find it helpful to: - - configure Git to use your preferred editor, with the EDITOR +- The body is written in prose, with full paragraphs; each paragraph should + be separated from the next by a single blank line. +- The body explains: + - why and how the change was made + - any manual testing you did in addition to running the automated tests + - any aspects of the commit that you think are questionable and + you'd like special attention applied to. +- If the commit makes performance improvements, you should generally + include some rough benchmarks showing that it actually improves the + performance. +- When you fix a GitHub issue, [mark that you've fixed the issue in + your commit + message](https://help.github.com/en/articles/closing-issues-via-commit-messages) + so that the issue is automatically closed when your code is merged. + Zulip's preferred style for this is to have the final paragraph of + the commit message read e.g. "Fixes: \#123.". +- Avoid `Partially fixes #1234`; GitHub's regular expressions ignore + the "partially" and close the issue. `Fixes part of #1234` is a good alternative. +- Any paragraph content in the commit message should be line-wrapped + to about 68 characters per line, but no more than 70, so that your + commit message will be reasonably readable in `git log` in a normal + terminal. You may find it helpful to: + - configure Git to use your preferred editor, with the EDITOR environment variable or `git config --global core.editor`, and - - configure the editor to automatically wrap text to 70 or fewer + - configure the editor to automatically wrap text to 70 or fewer columns per line (all text editors support this). diff --git a/docs/contributing/zulipbot-usage.md b/docs/contributing/zulipbot-usage.md index 85b80eaaa0..bae99d83fd 100644 --- a/docs/contributing/zulipbot-usage.md +++ b/docs/contributing/zulipbot-usage.md @@ -14,8 +14,8 @@ model GitHub supports). ## Usage - **Claim an issue** — Comment `@zulipbot claim` on the issue you want -to claim; **@zulipbot** will assign you to the issue and label the issue as -**in progress**. + to claim; **@zulipbot** will assign you to the issue and label the issue as + **in progress**. - If you're a new contributor, **@zulipbot** will give you read-only collaborator access to the repository and leave a welcome message on the @@ -28,8 +28,8 @@ to claim; **@zulipbot** will assign you to the issue and label the issue as `@zulipbot abandon` to abandon an issue. - **Label your issues** — Add appropriate labels to issues that you opened by -including `@zulipbot add` in an issue comment or the body of your issue -followed by the desired labels enclosed within double quotes (`""`). + including `@zulipbot add` in an issue comment or the body of your issue + followed by the desired labels enclosed within double quotes (`""`). - For example, to add the **bug** and **help wanted** labels to your issue, comment or include `@zulipbot add "bug" "help wanted"` in the @@ -43,8 +43,8 @@ followed by the desired labels enclosed within double quotes (`""`). (`""`). - **Find unclaimed issues** — Use the [GitHub search -feature](https://help.github.com/en/articles/using-search-to-filter-issues-and-pull-requests) -to find unclaimed issues by adding one of the following filters to your search: + feature](https://help.github.com/en/articles/using-search-to-filter-issues-and-pull-requests) + to find unclaimed issues by adding one of the following filters to your search: - `-label: "in progress"` (excludes issues labeled with the **in progress** label) @@ -54,19 +54,19 @@ to find unclaimed issues by adding one of the following filters to your search: already been claimed. - **Collaborate in area label teams** — Receive notifications on -issues and pull requests within your fields of expertise on the -[Zulip server repository](https://github.com/zulip/zulip) by joining -the Zulip server -[area label teams](https://github.com/orgs/zulip/teams?utf8=✓&query=Server) -(Note: this link only works for members of the Zulip organization; -we'll happily add you if you're interested). These teams correspond -to the repository's -[area labels](https://github.com/zulip/zulip/labels), although some -teams are associated with multiple labels; for example, the **area: -message-editing** and **area: message view** labels are both related -to the -[Server message view](https://github.com/orgs/zulip/teams/server-message-view) -team. Feel free to join as many area label teams as as you'd like! + issues and pull requests within your fields of expertise on the + [Zulip server repository](https://github.com/zulip/zulip) by joining + the Zulip server + [area label teams](https://github.com/orgs/zulip/teams?utf8=✓&query=Server) + (Note: this link only works for members of the Zulip organization; + we'll happily add you if you're interested). These teams correspond + to the repository's + [area labels](https://github.com/zulip/zulip/labels), although some + teams are associated with multiple labels; for example, the **area: + message-editing** and **area: message view** labels are both related + to the + [Server message view](https://github.com/orgs/zulip/teams/server-message-view) + team. Feel free to join as many area label teams as as you'd like! After your request to join an area label team is approved, you'll receive notifications for any issues labeled with the team's corresponding area @@ -74,8 +74,8 @@ team. Feel free to join as many area label teams as as you'd like! team's area label. - **Track inactive claimed issues** — If a claimed issue has not been updated -for a week, **@zulipbot** will post a comment on the inactive issue to ask the -assignee(s) if they are still working on the issue. + for a week, **@zulipbot** will post a comment on the inactive issue to ask the + assignee(s) if they are still working on the issue. If you see this comment on an issue you claimed, you should post a comment on the issue to notify **@zulipbot** that you're still working on it. diff --git a/docs/development/authentication.md b/docs/development/authentication.md index 86fa7733f0..5b3a1b2891 100644 --- a/docs/development/authentication.md +++ b/docs/development/authentication.md @@ -46,9 +46,9 @@ details worth understanding: ### Google - Visit [the Google developer -console](https://console.developers.google.com) and navigate to "APIs -& services" > "Credentials". Create a "Project", which will correspond -to your dev environment. + console](https://console.developers.google.com) and navigate to "APIs + & services" > "Credentials". Create a "Project", which will correspond + to your dev environment. - Navigate to "APIs & services" > "Library", and find the "Identity Toolkit API". Choose "Enable". @@ -96,12 +96,12 @@ to your dev environment. - [Create a Sign in with Apple private key](https://help.apple.com/developer-account/?lang=en#/dev77c875b7e) - In `dev-secrets.conf`, set - - `social_auth_apple_services_id` to your - "Services ID" (eg. com.application.your). - - `social_auth_apple_app_id` to "App ID" or "Bundle ID". - This is only required if you are testing Apple auth on iOS. - - `social_auth_apple_key` to your "Key ID". - - `social_auth_apple_team` to your "Team ID". + - `social_auth_apple_services_id` to your + "Services ID" (eg. com.application.your). + - `social_auth_apple_app_id` to "App ID" or "Bundle ID". + This is only required if you are testing Apple auth on iOS. + - `social_auth_apple_key` to your "Key ID". + - `social_auth_apple_team` to your "Team ID". - Put the private key file you got from apple at the path `zproject/dev_apple.key`. @@ -111,21 +111,21 @@ to your dev environment. - Set up SAML authentication by following [Okta's documentation](https://developer.okta.com/docs/guides/saml-application-setup/overview/). Specify: - - `http://localhost:9991/complete/saml/` for the "Single sign on URL"`. - - `http://localhost:9991` for the "Audience URI (SP Entity ID)". - - Skip "Default RelayState". - - Skip "Name ID format". - - Set 'Email` for "Application username format". - - Provide "Attribute statements" of `email` to `user.email`, - `first_name` to `user.firstName`, and `last_name` to `user.lastName`. + - `http://localhost:9991/complete/saml/` for the "Single sign on URL"`. + - `http://localhost:9991` for the "Audience URI (SP Entity ID)". + - Skip "Default RelayState". + - Skip "Name ID format". + - Set 'Email` for "Application username format". + - Provide "Attribute statements" of `email` to `user.email`, + `first_name` to `user.firstName`, and `last_name` to `user.lastName`. - Assign at least one account in the "Assignments" tab. You'll use it for signing up / logging in to Zulip. - Visit the big "Setup instructions" button on the "Sign on" tab. - Edit `zproject/dev-secrets.conf` to add the two values provided: - - Set `saml_url = http...` from "Identity Provider Single Sign-On - URL". - - Set `saml_entity_id = http://...` from "Identity Provider Issuer". - - Download the certificate and put it at the path `zproject/dev_saml.cert`. + - Set `saml_url = http...` from "Identity Provider Single Sign-On + URL". + - Set `saml_entity_id = http://...` from "Identity Provider Issuer". + - Download the certificate and put it at the path `zproject/dev_saml.cert`. - Now you should have working SAML authentication! - You can sign up to the target realm with the account that you've "assigned" in the previous steps (if the account's email address is allowed in the realm, @@ -159,12 +159,13 @@ development environment as well, so that you can go through all the actual flows for LDAP configuration. - To enable fakeldap, set `FAKE_LDAP_MODE` in -`zproject/dev_settings.py` to one of the following options. For more -information on these modes, refer to -[our production docs](../production/authentication-methods.html#ldap-including-active-directory): + `zproject/dev_settings.py` to one of the following options. For more + information on these modes, refer to + [our production docs](../production/authentication-methods.html#ldap-including-active-directory): + - `a`: If users' email addresses are in LDAP and used as username. - `b`: If LDAP only has usernames but email addresses are of the form - username@example.com + username@example.com - `c`: If LDAP usernames are completely unrelated to email addresses. - To disable fakeldap, set `FAKE_LDAP_MODE` back to `None`. @@ -174,8 +175,8 @@ information on these modes, refer to `ldapuser1`). - `FAKE_LDAP_NUM_USERS` in `zproject/dev_settings.py` can be used to -specify the number of LDAP users to be added. The default value for -the number of LDAP users is 8. + specify the number of LDAP users to be added. The default value for + the number of LDAP users is 8. ### Testing avatar and custom profile field synchronization diff --git a/docs/development/remote.md b/docs/development/remote.md index d99521c1af..d605f524e0 100644 --- a/docs/development/remote.md +++ b/docs/development/remote.md @@ -19,7 +19,7 @@ The best way to connect to your server is using the command line tool `ssh`. - On macOS and Linux/UNIX, `ssh` is a part of Terminal. - On Windows, `ssh` comes with [Bash for Git][git-bash]. -Open *Terminal* or *Bash for Git*, and connect with the following: +Open _Terminal_ or _Bash for Git_, and connect with the following: ```console $ ssh username@host @@ -37,14 +37,15 @@ to the next section. You can create a new user with sudo privileges by running the following commands as root: + - You can create a `zulipdev` user by running the command -`adduser zulipdev`. Run through the prompts to assign a password and -user information. (You can pick any username you like for this user -account.) + `adduser zulipdev`. Run through the prompts to assign a password and + user information. (You can pick any username you like for this user + account.) - You can add the user to the sudo group by running the command -`usermod -aG sudo zulipdev`. + `usermod -aG sudo zulipdev`. - Finally, you can switch to the user by running the command -`su - zulipdev` (or just log in to that user using `ssh`). + `su - zulipdev` (or just log in to that user using `ssh`). ## Setting up the development environment @@ -149,7 +150,7 @@ guide][rtd-git-guide]. In brief, the steps are as follows. On your **local computer**: -1. Open *Terminal* (macOS/Linux) or *Git for BASH*. +1. Open _Terminal_ (macOS/Linux) or _Git for BASH_. 2. Change directory to where you cloned Zulip (e.g. `cd zulip`). 3. Use `git add` and `git commit` to stage and commit your changes (if you haven't already). @@ -160,7 +161,7 @@ Be sure to replace `branchname` with the name of your actual feature branch. Once `git push` has completed successfully, you are ready to fetch the commits from your remote development instance: -1. In *Terminal* or *Git BASH*, connect to your remote development +1. In _Terminal_ or _Git BASH_, connect to your remote development instance with `ssh user@host`. 2. Change to the zulip directory (e.g., `cd zulip`). 3. Fetch new commits from GitHub with `git fetch origin`. @@ -193,12 +194,13 @@ Similar packages/extensions exist for other popular code editors as well; contributions of precise documentation for them are welcome! - [VSCode Remote - SSH][vscode-remote-ssh]: Lets you use Visual Studio -Code against a remote repository with a similar user experience to -developing locally. + Code against a remote repository with a similar user experience to + developing locally. [vscode-remote-ssh]: https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-ssh - [rmate](https://github.com/textmate/rmate) for TextMate + VS Code: + 1. Install the extension [Remote VSCode](https://marketplace.visualstudio.com/items?itemName=rafaelmaiolla.remote-vscode). 2. On your remote machine, run: @@ -229,11 +231,11 @@ Two editors often available by default on Linux systems are: - **Nano**: A very simple, beginner-friendly editor. However, it lacks a lot of features useful for programming, such as syntax highlighting, so we only recommended it for quick edits to things like configuration files. Launch by - running command `nano `. Exit by pressing *Ctrl-X*. + running command `nano `. Exit by pressing _Ctrl-X_. - **[Vim](https://www.vim.org/)**: A very powerful editor that can take a while - to learn. Launch by running `vim `. Quit Vim by pressing *Esc*, - typing `:q`, and then pressing *Enter*. Vim comes with a program to learn it + to learn. Launch by running `vim `. Quit Vim by pressing _Esc_, + typing `:q`, and then pressing _Enter_. Vim comes with a program to learn it called `vimtutor` (just run that command to start it). Other options include: @@ -250,12 +252,12 @@ started working quickly, we recommend web-based IDE To set up Codeanywhere for Zulip: 1. Create a [Codeanywhere][codeanywhere] account and log in. -2. Create a new **SFTP-SSH** project. Use *Public key* for authentication. +2. Create a new **SFTP-SSH** project. Use _Public key_ for authentication. 3. Click **GET YOUR PUBLIC KEY** to get the new public key that Codeanywhere generates when you create a new project. Add this public key to `~/.ssh/authorized_keys` on your remote development instance. 4. Once you've added the new public key to your remote development instance, click - *CONNECT*. + _CONNECT_. Now your workspace should look similar this: ![Codeanywhere workspace][img-ca-workspace] @@ -290,26 +292,27 @@ that the user is `zulipdev`; edit accordingly if the situation is different. 1. First, get an SSL certificate; you can use - [our certbot wrapper script used for production](../production/ssl-certificates.html#certbot-recommended) - by running the following commands as root: - ```bash - # apt install -y crudini - mkdir -p /var/lib/zulip/certbot-webroot/ - # if nginx running this will fail and you need to run `service nginx stop` - /home/zulipdev/zulip/scripts/setup/setup-certbot \ - hostname.example.com --no-zulip-conf \ - --email=username@example.com --method=standalone - ``` + [our certbot wrapper script used for production](../production/ssl-certificates.html#certbot-recommended) + by running the following commands as root: + + ```bash + # apt install -y crudini + mkdir -p /var/lib/zulip/certbot-webroot/ + # if nginx running this will fail and you need to run `service nginx stop` + /home/zulipdev/zulip/scripts/setup/setup-certbot \ + hostname.example.com --no-zulip-conf \ + --email=username@example.com --method=standalone + ``` 1. Install nginx configuration: - ```bash - apt install -y nginx-full - cp -a /home/zulipdev/zulip/tools/droplets/zulipdev /etc/nginx/sites-available/ - ln -nsf /etc/nginx/sites-available/zulipdev /etc/nginx/sites-enabled/ - nginx -t # Verifies your nginx configuration - service nginx reload # Actually enabled your nginx configuration - ``` + ```bash + apt install -y nginx-full + cp -a /home/zulipdev/zulip/tools/droplets/zulipdev /etc/nginx/sites-available/ + ln -nsf /etc/nginx/sites-available/zulipdev /etc/nginx/sites-enabled/ + nginx -t # Verifies your nginx configuration + service nginx reload # Actually enabled your nginx configuration + ``` 1. Edit `zproject/dev_settings.py` to set `EXTERNAL_URI_SCHEME = "https://"`, so that URLs served by the diff --git a/docs/development/setup-advanced.md b/docs/development/setup-advanced.md index 9214436ba1..ce877b40df 100644 --- a/docs/development/setup-advanced.md +++ b/docs/development/setup-advanced.md @@ -64,7 +64,7 @@ the [WSL 2](https://docs.microsoft.com/en-us/windows/wsl/wsl2-about) installation method described here. 1. Install WSL 2 by following the instructions provided by Microsoft -[here](https://docs.microsoft.com/en-us/windows/wsl/wsl2-install). + [here](https://docs.microsoft.com/en-us/windows/wsl/wsl2-install). 1. Install the `Ubuntu 18.04` Linux distribution from the Microsoft Store. @@ -92,6 +92,7 @@ installation method described here. 1. Make sure you are inside the WSL disk and not in a Windows mounted disk. You will run into permission issues if you run `provision` from `zulip` in a Windows mounted disk. + ```bash cd ~ # or cd /home/USERNAME ``` diff --git a/docs/development/setup-vagrant.md b/docs/development/setup-vagrant.md index 1a69f04410..c5675aa629 100644 --- a/docs/development/setup-vagrant.md +++ b/docs/development/setup-vagrant.md @@ -10,6 +10,7 @@ or a Linux container (for Ubuntu) inside which the Zulip server and all related services will run. Contents: + - [Requirements](#requirements) - [Step 0: Set up Git & GitHub](#step-0-set-up-git-github) - [Step 1: Install prerequisites](#step-1-install-prerequisites) @@ -159,11 +160,11 @@ Debian](https://docs.docker.com/install/linux/docker-ce/debian/). We recommend using `WSL 2 for Windows development <../development/setup-advanced.html#installing-directly-on-windows-10-with-wsl-2>`_. ``` -1. Install [Git for Windows][git-bash], which installs *Git BASH*. +1. Install [Git for Windows][git-bash], which installs _Git BASH_. 2. Install [VirtualBox][vbox-dl] (latest). 3. Install [Vagrant][vagrant-dl] (latest). -(Note: While *Git BASH* is recommended, you may also use [Cygwin][cygwin-dl]. +(Note: While _Git BASH_ is recommended, you may also use [Cygwin][cygwin-dl]. If you do, make sure to **install default required packages** along with **git**, **curl**, **openssh**, and **rsync** binaries.) @@ -363,6 +364,7 @@ Quit the server with CTRL-C. 2016-05-04 18:20:40,722 INFO Tornado 95.5% busy over the past 0.0 seconds Performing system checks... ``` + And ending with something similar to: ```console @@ -494,6 +496,7 @@ logout Connection to 127.0.0.1 closed. christie@win10 ~/zulip ``` + Now you can suspend the development environment: ```console @@ -537,8 +540,8 @@ Next, read the following to learn more about developing for Zulip: - [Git & GitHub guide][rtd-git-guide] - [Using the development environment][rtd-using-dev-env] - [Testing][rtd-testing] (and [Configuring CI][ci] to -run the full test suite against any branches you push to your fork, -which can help you optimize your development workflow). + run the full test suite against any branches you push to your fork, + which can help you optimize your development workflow). ### Troubleshooting and common errors @@ -590,6 +593,7 @@ shell and run `vagrant ssh` again to get the virtualenv setup properly. #### Vagrant was unable to mount VirtualBox shared folders For the following error: + ```console Vagrant was unable to mount VirtualBox shared folders. This is usually because the filesystem "vboxsf" is not available. This filesystem is @@ -810,6 +814,7 @@ proxy to access the Internet and haven't [configured the development environment to use it](#specifying-a-proxy). Once you've provisioned successfully, you'll get output like this: + ```console Zulip development environment setup succeeded! (zulip-py3-venv) vagrant@vagrant-base-trusty-amd64:~/zulip$ @@ -830,10 +835,10 @@ Likely causes are: 1. Networking issues 2. Insufficient RAM. Check whether you've allotted at least two -gigabytes of RAM, which is the minimum Zulip -[requires](../development/setup-vagrant.html#requirements). If -not, go to your VM settings and increase the RAM, then restart -the VM. + gigabytes of RAM, which is the minimum Zulip + [requires](../development/setup-vagrant.html#requirements). If + not, go to your VM settings and increase the RAM, then restart + the VM. ##### yarn install warnings @@ -904,7 +909,7 @@ vagrant ssh -- 'modinfo -F version vboxsf' The bug has not been fixed upstream as of this writing, but you may be able to work around it by downgrading VirtualBox Guest Additions to -6.0.4. To do this, create a `~/.zulip-vagrant-config` file and add +6.0.4. To do this, create a `~/.zulip-vagrant-config` file and add this line: ```text diff --git a/docs/development/test-install.md b/docs/development/test-install.md index 9a4dee8008..e08ac7b46d 100644 --- a/docs/development/test-install.md +++ b/docs/development/test-install.md @@ -1,12 +1,12 @@ # Testing the installer Zulip's install process is tested as part of [its continuous -integrations suite][CI], but that only tests the most common +integrations suite][ci], but that only tests the most common configurations; when making changes to more complicated [installation options][installer-docs], Zulip provides tooling to repeatedly test the installation process in a clean environment each time. -[CI]: https://github.com/zulip/zulip/actions/workflows/production-suite.yml?query=branch%3Amain +[ci]: https://github.com/zulip/zulip/actions/workflows/production-suite.yml?query=branch%3Amain [installer-docs]: ../production/install.md ## Configuring @@ -17,6 +17,7 @@ RAM, in order to accommodate the VMs and the steps which build the release assets. To begin, install the LXC toolchain: + ```bash sudo apt-get install lxc lxc-utils ``` @@ -32,6 +33,7 @@ You only need to do this step once per time you work on a set of changes, to refresh the package that the installer uses. The installer doesn't work cleanly out of a source checkout; it wants a release checkout, so we build a tarball of one of those first: + ```bash ./tools/build-release-tarball test-installer ``` @@ -46,6 +48,7 @@ directory. The test installer needs the release directory to be named `zulip-server`, so we rename it and move it appropriately. In the first line, you'll need to substitute the actual path that you got for the tarball, above: + ```bash tar xzf /tmp/tmp.fepqqNBWxp/zulip-server-test-installer.tar.gz mkdir zulip-test-installer @@ -65,6 +68,7 @@ into the installer. For example, to test an install onto Ubuntu 20.04 "Focal", we might call: + ```bash sudo ./tools/test-install/install \ -r focal \ @@ -82,6 +86,7 @@ take a while. Regardless of if the install succeeds or fails, it will stay running so you can inspect it. You can see all of the containers which are running, and their randomly-generated names, by running: + ```bash sudo lxc-ls -f ``` @@ -90,6 +95,7 @@ sudo lxc-ls -f After using `lxc-ls` to list containers, you can choose one of them and connect to its terminal: + ```bash sudo lxc-attach --clear-env -n zulip-install-focal-PUvff ``` @@ -98,23 +104,24 @@ sudo lxc-attach --clear-env -n zulip-install-focal-PUvff To destroy all containers (but leave the base containers, which speed up the initial install): + ```bash sudo ./tools/test-install/destroy-all -f ``` To destroy just one container: + ```bash sudo lxc-destroy -f -n zulip-install-focal-PUvff ``` - - ### Iterating on the installer Iterate on the installer by making changes to your source tree, copying them into the release directory, and re-running the installer, which will start up a new container. Here, we update just the `scripts` and `puppet` directories of the release directory: + ```bash rsync -az scripts puppet zulip-test-installer/zulip-server/ diff --git a/docs/development/using.md b/docs/development/using.md index 5dc5582ecc..d140bd7133 100644 --- a/docs/development/using.md +++ b/docs/development/using.md @@ -1,5 +1,4 @@ -Using the development environment -================================= +# Using the development environment This page describes the basic edit/refresh workflows for working with the Zulip development environment. Generally, the development @@ -40,12 +39,12 @@ the development environment][authentication-dev-server]. - For changes that don't affect the database model, the Zulip development environment will automatically detect changes and restart: - - The main Django/Tornado server processes are run on top of + - The main Django/Tornado server processes are run on top of Django's [manage.py runserver][django-runserver], which will automatically restart them when you save changes to Python code they use. You can watch this happen in the `run-dev.py` console to make sure the backend has reloaded. - - The Python queue workers will also automatically restart when you + - The Python queue workers will also automatically restart when you save changes, as long as they haven't crashed (which can happen if they reloaded into a version with a syntax error). - If you change the database schema (`zerver/models.py`), you'll need @@ -69,9 +68,9 @@ the development environment][authentication-dev-server]. - By default, the development server homepage just shows a list of the users that exist on the server and you can log in as any of them by just clicking on a user. - - This setup saves time for the common case where you want to test + - This setup saves time for the common case where you want to test something other than the login process. - - You can test the login or registration process by clicking the + - You can test the login or registration process by clicking the links for the normal login page. - Most changes will take effect automatically. Details: - If you change CSS files, your changes will appear immediately via diff --git a/docs/documentation/api.md b/docs/documentation/api.md index 302374e9d1..d2c9123028 100644 --- a/docs/documentation/api.md +++ b/docs/documentation/api.md @@ -124,7 +124,7 @@ For the Python examples, you'll write the example in automatically in Zulip's automated test suite. The code there will look something like this: -``` python +```python @openapi_test_function('/messages/render:post') def render_message(client: Client) -> None: # {code_example|start} @@ -215,10 +215,11 @@ above. declared using `REQ`. You can check your formatting using these helpful tools. + - `tools/check-openapi` will verify the syntax of `zerver/openapi/zulip.yaml`. - `tools/test-backend zerver/tests/test_openapi.py`; this test compares - your documentation against the code and can find many common - mistakes in how arguments are declared. + your documentation against the code and can find many common + mistakes in how arguments are declared. - `test-backend`: The full Zulip backend test suite will fail if any actual API responses generated by the tests don't match your defined OpenAPI schema. Use `test-backend --rerun` for a fast diff --git a/docs/documentation/integrations.md b/docs/documentation/integrations.md index 09371f0cde..844c94cbbe 100644 --- a/docs/documentation/integrations.md +++ b/docs/documentation/integrations.md @@ -84,12 +84,12 @@ Here are a few common macros used to document Zulip's integrations: This macro is usually used right after `{!create-stream!}`. For an example rendering, see **Step 2** of [the docs for Zulip's GitHub integration][github-integration]. - **Note:** If special configuration is - required to set up the URL and you can't use this macro, be sure to use the - `{{ api_url }}` template variable, so that your integration - documentation will provide the correct URL for whatever server it is - deployed on. If special configuration is required to set the `SITE` - variable, you should document that too. + **Note:** If special configuration is + required to set up the URL and you can't use this macro, be sure to use the + `{{ api_url }}` template variable, so that your integration + documentation will provide the correct URL for whatever server it is + deployed on. If special configuration is required to set the `SITE` + variable, you should document that too. - `{!append-stream-name.md!}` macro - Recommends appending `&stream=stream_name` to a URL in cases where supplying a stream name in the URL is optional. @@ -136,12 +136,12 @@ Here are a few common macros used to document Zulip's integrations: - `{!webhook-url-with-bot-email.md!}` - Used in certain non-webhook integrations to generate URLs of the form: - ```text - https://bot_email:bot_api_key@yourZulipDomain.zulipchat.com/api/v1/external/beanstalk - ``` + ```text + https://bot_email:bot_api_key@yourZulipDomain.zulipchat.com/api/v1/external/beanstalk + ``` - For an example rendering, see - [Zulip's Beanstalk integration](https://zulip.com/integrations/doc/beanstalk). + For an example rendering, see + [Zulip's Beanstalk integration](https://zulip.com/integrations/doc/beanstalk). [github-integration]: https://zulip.com/integrations/doc/github [codebase]: https://zulip.com/integrations/doc/codebase @@ -189,7 +189,6 @@ concrete guidelines. - Follow the organization and wording of existing docs as much as possible. - ### Guidelines for specific steps Most doc files should start with a generic sentence about the diff --git a/docs/documentation/openapi.md b/docs/documentation/openapi.md index 2d9cd18973..634e1b3525 100644 --- a/docs/documentation/openapi.md +++ b/docs/documentation/openapi.md @@ -40,6 +40,7 @@ types of authentication, and configure other settings. Once defined, information in this section rarely changes. For example, the `swagger` and `info` objects look like this: + ```yaml # Basic Swagger UI info openapi: 3.0.1 diff --git a/docs/documentation/overview.md b/docs/documentation/overview.md index e8cec5c15c..0b5dd4ca58 100644 --- a/docs/documentation/overview.md +++ b/docs/documentation/overview.md @@ -18,11 +18,11 @@ Zulip has three major documentation systems: several hundred pages written using this system. There are 3 branches of this documentation: - User documentation (with a target audience of individual Zulip - users), + users), - Integrations documentation (with a target audience of IT folks - setting up integrations), and + setting up integrations), and - API documentation (with a target audience of developers writing - code to extend Zulip). + code to extend Zulip). These three systems are documented in detail. @@ -124,21 +124,21 @@ Zulip has several automated test suites that we run in CI and recommend running locally when making significant edits: - `tools/lint` catches a number of common mistakes, and we highly -recommend -[using our linter pre-commit hook](../git/zulip-tools.html#set-up-git-repo-script). -See the [main linter doc](../testing/linters.md) for more details. + recommend + [using our linter pre-commit hook](../git/zulip-tools.html#set-up-git-repo-script). + See the [main linter doc](../testing/linters.md) for more details. - The ReadTheDocs docs are built and the links tested by -`tools/test-documentation`, which runs `build-docs` and then checks -all the links. + `tools/test-documentation`, which runs `build-docs` and then checks + all the links. There's an exclude list for the link testing at this horrible path: `tools/documentation_crawler/documentation_crawler/spiders/common/spiders.py`, which is relevant for flaky links. - The API docs are tested by `tools/test-api`, which does some basic -payload verification. Note that this test does not check for broken -links (those are checked by `test-help-documentation`). + payload verification. Note that this test does not check for broken + links (those are checked by `test-help-documentation`). - `tools/test-help-documentation` checks `/help/`, `/api/`, `/integrations/`, and the core website ("portico") documentation for diff --git a/docs/documentation/user.md b/docs/documentation/user.md index b1ceff918a..f5ebf9930d 100644 --- a/docs/documentation/user.md +++ b/docs/documentation/user.md @@ -8,6 +8,7 @@ There are two types of documents: articles about specific features, and a handful of longer guides. The feature articles serve a few different purposes: + - Feature discovery, for someone browsing the `/help` page, and looking at the set of titles. - Public documentation of our featureset, for someone googling "can zulip do .." @@ -29,7 +30,7 @@ ReadTheDocs, since Zulip supports running a server completely disconnected from the Internet, and we'd like the documentation to be available in that environment. - The source for this user documentation is the Markdown files under +The source for this user documentation is the Markdown files under `templates/zerver/help/` in the [main Zulip server repository](https://github.com/zulip/zulip). The file `foo.md` is automatically rendered by the `render_markdown_path` function in @@ -99,15 +100,15 @@ Zulip's Markdown processor allows you to include several special features in your documentation to help improve its readability: - Since raw HTML is supported in Markdown, you can include arbitrary -HTML/CSS in your documentation as needed. + HTML/CSS in your documentation as needed. - Code blocks allow you to highlight syntax, similar to Zulip's own Markdown. - Anchor tags can be used to link to headers in other documents. - [Images](#images) of Zulip UI can be added to documentation. - Inline [icons](#icons) used to refer to features in the Zulip UI. - You can utilize [macros](#macros) to limit repeated content in the -documentation. + documentation. - You can create special highlight warning blocks using -[tips and warnings](#tips-and-warnings). + [tips and warnings](#tips-and-warnings). - You can create tabs using [Markdown tab switcher](#tab-switcher). ### Images @@ -127,7 +128,7 @@ instructions for something simple look long and complicated. When taking screenshots, the image should never include the whole Zulip browser window in a screenshot; instead, it should only show relevant parts of the app. In addition, the screenshot should always -come *after* the text that describes it, never before. +come _after_ the text that describes it, never before. Images are often a part of a numbered step and must be indented four spaces to be formatted correctly. @@ -142,39 +143,39 @@ icons from [FontAwesome](https://fontawesome.com/v4.7.0/) (version 4.7.0) which make use of `fa` as a base class. - cog () icon — -`cog () icon` + `cog () icon` - down chevron () icon — -`down chevron () icon` + `down chevron () icon` - eye () icon — -`eye () icon` + `eye () icon` - file () icon — -`file () icon` + `file () icon` - filled star () icon — -`filled star () icon` + `filled star () icon` - formatting () icon — -`formatting () icon` + `formatting () icon` - menu () icon — -`menu () icon` + `menu () icon` - overflow ( ) icon — -`overflow ( ) icon` + `overflow ( ) icon` - paperclip () icon — -`paperclip () icon` + `paperclip () icon` - pencil () icon — -`pencil () icon` + `pencil () icon` - pencil and paper () icon — -`pencil and paper () icon` + `pencil and paper () icon` - plus () icon — -`plus () icon` + `plus () icon` - smiley face () icon — -`smiley face () icon` + `smiley face () icon` - star () icon — -`star () icon` + `star () icon` - trash () icon — -`trash () icon` + `trash () icon` - video-camera () icon — -`video-camera () icon` + `video-camera () icon` - x () icon — -`x () icon` + `x () icon` ### Macros @@ -230,7 +231,7 @@ All tips/warnings should appear inside tip/warning blocks. There should be only one tip/warning inside each block, and they usually should be formatted as a continuation of a numbered step. -### Tab switcher +### Tab switcher Our Markdown processor supports easily creating a tab switcher widget design to easily show the instructions for different diff --git a/docs/git/cheat-sheet.md b/docs/git/cheat-sheet.md index 0cb34467c1..16910b881a 100644 --- a/docs/git/cheat-sheet.md +++ b/docs/git/cheat-sheet.md @@ -5,111 +5,111 @@ See also [fixing commits][fix-commit] ## Common commands - add - - `git add foo.py` + - `git add foo.py` - checkout - - `git checkout -b new-branch-name` - - `git checkout main` - - `git checkout old-branch-name` + - `git checkout -b new-branch-name` + - `git checkout main` + - `git checkout old-branch-name` - commit - - `git commit -m "topic: Commit message title."` - - `git commit --amend`: Modify the previous commit. + - `git commit -m "topic: Commit message title."` + - `git commit --amend`: Modify the previous commit. - config - - `git config --global core.editor nano` - - `git config --global core.symlinks true` + - `git config --global core.editor nano` + - `git config --global core.symlinks true` - diff - - `git diff` - - `git diff --cached` - - `git diff HEAD~2..` + - `git diff` + - `git diff --cached` + - `git diff HEAD~2..` - fetch - - `git fetch origin` - - `git fetch upstream` + - `git fetch origin` + - `git fetch upstream` - grep - - `git grep update_unread_counts` + - `git grep update_unread_counts` - log - - `git log` + - `git log` - pull - - `git pull --rebase`: **Use this**. Zulip uses a [rebase oriented workflow][git-overview]. - - `git pull` (with no options): Will either create a merge commit - (which you don't want) or do the same thing as `git pull --rebase`, - depending on [whether you've configured Git properly][git-config-clone] + - `git pull --rebase`: **Use this**. Zulip uses a [rebase oriented workflow][git-overview]. + - `git pull` (with no options): Will either create a merge commit + (which you don't want) or do the same thing as `git pull --rebase`, + depending on [whether you've configured Git properly][git-config-clone] - push - - `git push origin +branch-name` + - `git push origin +branch-name` - rebase - - `git rebase -i HEAD~3` - - `git rebase -i main` - - `git rebase upstream/main` + - `git rebase -i HEAD~3` + - `git rebase -i main` + - `git rebase upstream/main` - reflog - - `git reflog | head -10` + - `git reflog | head -10` - remote - - `git remote -v` + - `git remote -v` - reset - - `git reset HEAD~2` + - `git reset HEAD~2` - rm - - `git rm oops.txt` + - `git rm oops.txt` - show - - `git show HEAD` - - `git show HEAD~~~` - - `git show main` + - `git show HEAD` + - `git show HEAD~~~` + - `git show main` - status - - `git status` + - `git status` ## Detailed cheat sheet - add - - `git add foo.py`: add `foo.py` to the staging area - - `git add foo.py bar.py`: add `foo.py` AND `bar.py` to the staging area - - `git add -u`: Adds all tracked files to the staging area. + - `git add foo.py`: add `foo.py` to the staging area + - `git add foo.py bar.py`: add `foo.py` AND `bar.py` to the staging area + - `git add -u`: Adds all tracked files to the staging area. - checkout - - `git checkout -b new-branch-name`: create branch `new-branch-name` and switch to/check out that new branch - - `git checkout main`: switch to your `main` branch - - `git checkout old-branch-name`: switch to an existing branch `old-branch-name` + - `git checkout -b new-branch-name`: create branch `new-branch-name` and switch to/check out that new branch + - `git checkout main`: switch to your `main` branch + - `git checkout old-branch-name`: switch to an existing branch `old-branch-name` - commit - - `git commit -m "commit message"`: It is recommended to type a - multiline commit message, however. - - `git commit`: Opens your default text editor to write a commit message. - - `git commit --amend`: changing the last commit message. Read more [here][fix-commit] + - `git commit -m "commit message"`: It is recommended to type a + multiline commit message, however. + - `git commit`: Opens your default text editor to write a commit message. + - `git commit --amend`: changing the last commit message. Read more [here][fix-commit] - config - - `git config --global core.editor nano`: set core editor to `nano` (you can set this to `vim` or others) - - `git config --global core.symlinks true`: allow symbolic links + - `git config --global core.editor nano`: set core editor to `nano` (you can set this to `vim` or others) + - `git config --global core.symlinks true`: allow symbolic links - diff - - `git diff`: display the changes you have made to all files - - `git diff --cached`: display the changes you have made to staged files - - `git diff HEAD~2..`: display the 2 most recent changes you have made to files + - `git diff`: display the changes you have made to all files + - `git diff --cached`: display the changes you have made to staged files + - `git diff HEAD~2..`: display the 2 most recent changes you have made to files - fetch - - `git fetch origin`: fetch origin repository - - `git fetch upstream`: fetch upstream repository + - `git fetch origin`: fetch origin repository + - `git fetch upstream`: fetch upstream repository - grep - - `git grep update_unread_counts static/js`: Search our JS for references to update_unread_counts. + - `git grep update_unread_counts static/js`: Search our JS for references to update_unread_counts. - log - - `git log`: show commit logs - - `git log --oneline | head`: To quickly see the latest ten commits on a branch. + - `git log`: show commit logs + - `git log --oneline | head`: To quickly see the latest ten commits on a branch. - pull - - `git pull --rebase`: rebase your changes on top of `main`. - - `git pull` (with no options): Will either create a merge commit - (which you don't want) or do the same thing as `git pull --rebase`, - depending on [whether you've configured Git properly][git-config-clone] + - `git pull --rebase`: rebase your changes on top of `main`. + - `git pull` (with no options): Will either create a merge commit + (which you don't want) or do the same thing as `git pull --rebase`, + depending on [whether you've configured Git properly][git-config-clone] - push - - `git push origin branch-name`: push you commits to the origin repository *only if* there are no conflicts. - Use this when collaborating with others to prevent overwriting their work. - - `git push origin +branch-name`: force push your commits to your origin repository. + - `git push origin branch-name`: push you commits to the origin repository _only if_ there are no conflicts. + Use this when collaborating with others to prevent overwriting their work. + - `git push origin +branch-name`: force push your commits to your origin repository. - rebase - - `git rebase -i HEAD~3`: interactive rebasing current branch with first three items on HEAD - - `git rebase -i main`: interactive rebasing current branch with `main` branch - - `git rebase upstream/main`: rebasing current branch with `main` branch from upstream repository + - `git rebase -i HEAD~3`: interactive rebasing current branch with first three items on HEAD + - `git rebase -i main`: interactive rebasing current branch with `main` branch + - `git rebase upstream/main`: rebasing current branch with `main` branch from upstream repository - reflog - - `git reflog | head -10`: manage reference logs for the past 10 commits + - `git reflog | head -10`: manage reference logs for the past 10 commits - remote - - `git remote -v`: display your origin and upstream repositories + - `git remote -v`: display your origin and upstream repositories - reset - - `git reset HEAD~2`: reset two most recent commits + - `git reset HEAD~2`: reset two most recent commits - rm - - `git rm oops.txt`: remove `oops.txt` + - `git rm oops.txt`: remove `oops.txt` - show - - `git show HEAD`: display most recent commit - - `git show HEAD~~~`: display third most recent commit - - `git show main`: display most recent commit on `main` + - `git show HEAD`: display most recent commit + - `git show HEAD~~~`: display third most recent commit + - `git show main`: display most recent commit on `main` - status - - `git status`: show the working tree status, unstaged and staged files + - `git status`: show the working tree status, unstaged and staged files [fix-commit]: fixing-commits.md [git-config-clone]: cloning.html#step-1b-clone-to-your-machine diff --git a/docs/git/collaborate.md b/docs/git/collaborate.md index 4d2ec0a100..a7b8e54bcd 100644 --- a/docs/git/collaborate.md +++ b/docs/git/collaborate.md @@ -20,6 +20,7 @@ $ git checkout -b / ``` You can choose to rename the branch if you prefer: + ```bash git checkout -b / ``` @@ -31,8 +32,8 @@ pull request locally. GitHub provides a special syntax ([details][github-help-co-pr-locally]) for this since pull requests are specific to GitHub rather than Git. -First, fetch and create a branch for the pull request, replacing *ID* and -*BRANCHNAME* with the ID of the pull request and your desired branch name: +First, fetch and create a branch for the pull request, replacing _ID_ and +_BRANCHNAME_ with the ID of the pull request and your desired branch name: ```console $ git fetch upstream pull/ID/head:BRANCHNAME @@ -47,11 +48,12 @@ $ git checkout BRANCHNAME Now you work on this branch as you would any other. Note: you can use the scripts provided in the tools/ directory to fetch pull -requests. You can read more about what they do [here][tools-PR]. +requests. You can read more about what they do [here][tools-pr]. + ```bash tools/fetch-rebase-pull-request tools/fetch-pull-request ``` [github-help-co-pr-locally]: https://help.github.com/en/articles/checking-out-pull-requests-locally -[tools-PR]: ../git/zulip-tools.html#fetch-a-pull-request-and-rebase +[tools-pr]: ../git/zulip-tools.html#fetch-a-pull-request-and-rebase diff --git a/docs/git/fixing-commits.md b/docs/git/fixing-commits.md index c4f46d388b..018d9ae34e 100644 --- a/docs/git/fixing-commits.md +++ b/docs/git/fixing-commits.md @@ -1,35 +1,45 @@ # Fixing commits + This is mostly from [here](https://help.github.com/en/articles/changing-a-commit-message#rewriting-the-most-recent-commit-message). ## Fixing the last commit + ### Changing the last commit message + 1. `git commit --amend -m "New message"` ### Changing the last commit + 1. Make your changes to the files 2. Run `git add ` to add one file or `git add ...` to add multiple files 3. `git commit --amend` ## Fixing older commits + ### Changing commit messages + 1. `git rebase -i HEAD~5` (if, for example, you are editing some of the last five commits) 2. For each commit that you want to change the message, change `pick` to `reword`, and save 3. Change the commit messages ### Deleting old commits + 1. `git rebase -i HEAD~n` where `n` is the number of commits you are looking at 2. For each commit that you want to delete, change `pick` to `drop`, and save ## Squashing commits + Sometimes, you want to make one commit out of a bunch of commits. To do this, 1. `git rebase -i HEAD~n` where `n` is the number of commits you are interested in 2. Change `pick` to `squash` on the lines containing the commits you want to squash and save ## Reordering commits + 1. `git rebase -i HEAD~n` where `n` is the number of commits you are interested in 2. Reorder the lines containing the commits and save ## Pushing commits after tidying them + 1. `git push origin +my-feature-branch` (Note the `+` there and substitute your actual branch name.) diff --git a/docs/git/overview.md b/docs/git/overview.md index a3c4d8eed4..d70e531ef2 100644 --- a/docs/git/overview.md +++ b/docs/git/overview.md @@ -10,7 +10,7 @@ with these details in mind: [repository][github-zulip], if you are working on something else besides Zulip server) to your own account and then create feature/issue branches. When you're ready to get feedback, submit a work-in-progress (WIP) pull - request. *We encourage you to submit WIP pull requests early and often.* + request. _We encourage you to submit WIP pull requests early and often._ - We use a **[rebase][gitbook-rebase]-oriented workflow.** We do not use merge commits. This means you should use `git fetch` followed by `git rebase` @@ -24,13 +24,13 @@ with these details in mind: when another branch is merged, that clutter the commit history (it's popular with other large projects such as Django). This makes Zulip's commit history more readable, but a side effect is that many - pull requests we merge will be reported by GitHub's UI as *closed* - instead of *merged*, since GitHub has poor support for + pull requests we merge will be reported by GitHub's UI as _closed_ + instead of _merged_, since GitHub has poor support for rebase-oriented workflows. - We have a **[code style guide][zulip-rtd-code-style]**, a **[commit message - guide][zulip-rtd-commit-messages]**, and strive for each commit to be *a - minimal coherent idea* (see **[commit + guide][zulip-rtd-commit-messages]**, and strive for each commit to be _a + minimal coherent idea_ (see **[commit discipline][zulip-rtd-commit-discipline]** for details). - We provide **many tools to help you submit quality code.** These include @@ -48,7 +48,7 @@ with these details in mind: Finally, install the [Zulip developer environment][zulip-rtd-dev-overview], and then [configure continuous integration for your fork][zulip-git-guide-fork-ci]. -*** +--- The following sections will help you be awesome with Zulip and Git/GitHub in a rebased-based workflow. Read through it if you're new to Git, to a rebase-based diff --git a/docs/git/pull-requests.md b/docs/git/pull-requests.md index 183275660b..3dba7536de 100644 --- a/docs/git/pull-requests.md +++ b/docs/git/pull-requests.md @@ -3,7 +3,7 @@ When you're ready for feedback, submit a pull request. Pull requests are a feature specific to GitHub. They provide a simple, web-based way to submit your work (often called "patches") to a project. It's called -a *pull request* because you're asking the project to *pull changes* +a _pull request_ because you're asking the project to _pull changes_ from your fork. If you're unfamiliar with how to create a pull request, you can check @@ -44,7 +44,7 @@ The best way to update your branch is with `git fetch` and `git rebase`. Do not use `git pull` or `git merge` as this will create merge commits. See [keep your fork up to date][keep-up-to-date] for details. -Here's an example (you would replace *issue-123* with the name of your feature branch): +Here's an example (you would replace _issue-123_ with the name of your feature branch): ```console $ git checkout issue-123 @@ -117,7 +117,7 @@ pull request** button. Alternatively, if you've recently pushed to your fork, you will see a green **Compare & pull request** button. -You'll see the *Open a pull request* page: +You'll see the _Open a pull request_ page: ![images-create-pr] diff --git a/docs/git/terminology.md b/docs/git/terminology.md index a0090532f4..d8b549b1b3 100644 --- a/docs/git/terminology.md +++ b/docs/git/terminology.md @@ -2,9 +2,10 @@ When you install Git, it adds a manual entry for `gitglossary`. You can view this glossary by running `man gitglossary`. Below we've included the Git terms -you'll encounter most often along with their definitions from *gitglossary*. +you'll encounter most often along with their definitions from _gitglossary_. ## branch + A "branch" is an active line of development. The most recent commit on a branch is referred to as the tip of that branch. The tip of the branch is referenced by a branch head, which moves forward as @@ -14,14 +15,17 @@ working tree is associated with just one of them (the "current" or "checked out" branch), and HEAD points to that branch. ## cache + Obsolete for: index ## checkout + The action of updating all or part of the working tree with a tree object or blob from the object database, and updating the index and HEAD if the whole working tree has been pointed at a new branch. ## commit + As a noun: A single point in the Git history; the entire history of a project is represented as a set of interrelated commits. The word "commit" is often used by Git in the same places other revision @@ -33,6 +37,7 @@ state in the Git history, by creating a new commit representing the current state of the index and advancing HEAD to point at the new ## fast-forward + A fast-forward is a special type of merge where you have a revision and you are "merging" another branch's changes that happen to be a descendant of what you have. In such these cases, you do not make a @@ -41,19 +46,23 @@ happen frequently on a remote-tracking branch of a remote repository. ## fetch + Fetching a branch means to get the branch's head ref from a remote repository, to find out which objects are missing from the local object database, and to get them, too. See also [git-fetch(1)](https://git-scm.com/docs/git-fetch) ## hash + In Git's context, synonym for object name. ## head + A named reference to the commit at the tip of a branch. Heads are stored in a file in $GIT_DIR/refs/heads/ directory, except when using packed refs. See also [git-pack-refs(1)](https://git-scm.com/docs/git-pack-refs). ## HEAD + The current branch. In more detail: Your working tree is normally derived from the state of the tree referred to by HEAD. HEAD is a reference to one of the heads in your repository, except when using @@ -61,15 +70,18 @@ a detached HEAD, in which case it directly references an arbitrary commit. ## index + A collection of files with stat information, whose contents are stored as objects. The index is a stored version of your working tree. Truth be told, it can also contain a second, and even a third version of a working tree, which are used when merging. ## pull + Pulling a branch means to fetch it and merge it. See also [git-pull(1)](https://git-scm.com/docs/git-pull) ## push + Pushing a branch means to get the branch's head ref from a remote repository, find out if it is a direct ancestor to the branch's local head ref, and in that case, putting all objects, which are @@ -79,5 +91,6 @@ the remote head ref. If the remote head is not an ancestor to the local head, the push fails. ## rebase + To reapply a series of changes from a branch to a different base, and reset the head of that branch to the result. diff --git a/docs/git/the-git-difference.md b/docs/git/the-git-difference.md index 304eb7f423..e2569d0efd 100644 --- a/docs/git/the-git-difference.md +++ b/docs/git/the-git-difference.md @@ -2,22 +2,22 @@ Whether you're new to Git or have experience with another version control system (VCS), it's a good idea to learn a bit about how Git works. We recommend -this excellent presentation *[Understanding Git][understanding-git]* from +this excellent presentation _[Understanding Git][understanding-git]_ from Nelson Elhage and Anders Kaseorg and the [Git Basics][gitbook-basics] chapter -from *Pro Git* by Scott Chacon and Ben Straub. +from _Pro Git_ by Scott Chacon and Ben Straub. Here are the top things to know: - **Git works on snapshots.** Unlike other version control systems (e.g., Subversion, Perforce, Bazaar), which track files and changes to those files - made over time, Git tracks *snapshots* of your project. Each time you commit + made over time, Git tracks _snapshots_ of your project. Each time you commit or otherwise make a change to your repository, Git takes a snapshot of your project and stores a reference to that snapshot. If a file hasn't changed, Git creates a link to the identical file rather than storing it again. - **Most Git operations are local.** Git is a distributed version control system, so once you've cloned a repository, you have a complete copy of that - repository's *entire history*. Staging, committing, branching, and browsing + repository's _entire history_. Staging, committing, branching, and browsing history are all things you can do locally without network access and without immediately affecting any remote repositories. To make or receive changes from remote repositories, you need to `git fetch`, `git pull`, or `git push`. @@ -45,9 +45,9 @@ Here are the top things to know: - **Cloning a repository creates a working copy.** Every working copy has a `.git` subdirectory, which contains its own Git repository. The `.git` - subdirectory also tracks the *index*, a staging area for changes that will - become part of the next commit. All files outside of `.git` is the *working - tree*. + subdirectory also tracks the _index_, a staging area for changes that will + become part of the next commit. All files outside of `.git` is the _working + tree_. - **Files tracked with Git have possible three states: committed, modified, and staged.** Committed files are those safely stored in your local `.git` @@ -56,8 +56,8 @@ Here are the top things to know: changes but have not yet been marked for inclusion in the next commit; they have not been added to the index. -- **Git commit workflow is as follows.** Edit files in your *working tree*. Add - to the *index* (that is *stage*) with `git add`. *Commit* to the HEAD of the +- **Git commit workflow is as follows.** Edit files in your _working tree_. Add + to the _index_ (that is _stage_) with `git add`. _Commit_ to the HEAD of the current branch with `git commit`. [gitbook-basics]: https://git-scm.com/book/en/v2/Getting-Started-Git-Basics diff --git a/docs/git/troubleshooting.md b/docs/git/troubleshooting.md index 25fe0b7dbb..91f5011da4 100644 --- a/docs/git/troubleshooting.md +++ b/docs/git/troubleshooting.md @@ -204,10 +204,12 @@ and `>>>>>>>`) markers to indicate where in files there are conflicts. Tip: You can see recent changes made to a file by running the following commands: + ```bash git fetch upstream git log -p upstream/main -- /path/to/file ``` + You can use this to compare the changes that you have made to a file with the ones in upstream, helping you avoid undoing changes from a previous commit when you are rebasing. @@ -234,8 +236,8 @@ pay attention and do a bit of work to ensure all of your work is readily available. Recall that most Git operations are local. When you commit your changes with -`git commit` they are safely stored in your *local* Git database only. That is, -until you *push* the commits to GitHub, they are only available on the computer +`git commit` they are safely stored in your _local_ Git database only. That is, +until you _push_ the commits to GitHub, they are only available on the computer where you committed them. So, before you stop working for the day, or before you switch computers, push diff --git a/docs/git/using.md b/docs/git/using.md index 8c9f090714..b8653eb12e 100644 --- a/docs/git/using.md +++ b/docs/git/using.md @@ -135,7 +135,7 @@ Recall that files tracked with Git have possible three states: committed, modified, and staged. To prepare a commit, first add the files with changes that you want -to include in your commit to your staging area. You *add* both new files and +to include in your commit to your staging area. You _add_ both new files and existing ones. You can also remove files from staging when necessary. ### Get status of working directory @@ -168,7 +168,7 @@ nothing added to commit but untracked files present (use "git add" to track) To add changes to your staging area, use `git add `. Because `git add` is all about staging the changes you want to commit, you use -it to add *new files* as well as *files with changes* to your staging +it to add _new files_ as well as _files with changes_ to your staging area. Continuing our example from above, after we run `git add newfile.py`, we'll see @@ -188,7 +188,6 @@ view changes to files you haven't yet staged, just use `git diff`. If you want to add all changes in the working directory, use `git add -A` ([documentation][gitbook-add]). - You can also stage changes using your graphical Git client. If you stage a file, you can undo it with `git reset HEAD `. Here's diff --git a/docs/git/zulip-tools.md b/docs/git/zulip-tools.md index 2e5a2172f3..4a1148e0d1 100644 --- a/docs/git/zulip-tools.md +++ b/docs/git/zulip-tools.md @@ -41,7 +41,7 @@ described above in that it does not create a branch for the pull request checkout. **This tool checks for uncommitted changes, but it will move the - current branch using `git reset --hard`. Use with caution.** +current branch using `git reset --hard`. Use with caution.** First, make sure you are working in a branch you want to move (in this example, we'll use the local `main` branch). Then run the script @@ -163,10 +163,11 @@ Deleting local branch review-original-5156 (was 5a1e982) ## Merge conflict on yarn.lock file If there is a merge conflict on yarn.lock, yarn should be run to -regenerate the file. *Important* don't delete the yarn.lock file. Check out the +regenerate the file. _Important_ don't delete the yarn.lock file. Check out the latest one from `origin/main` so that yarn knows the previous asset versions. Run the following commands + ```bash git checkout origin/main -- yarn.lock yarn install diff --git a/docs/overview/architecture-overview.md b/docs/overview/architecture-overview.md index 4886aed66d..f15ac18cdd 100644 --- a/docs/overview/architecture-overview.md +++ b/docs/overview/architecture-overview.md @@ -1,8 +1,6 @@ -Zulip architectural overview -============================ +# Zulip architectural overview -Key codebases -------------- +## Key codebases The main Zulip codebase is at . It contains the Zulip backend (written in Python 3.x and Django), the @@ -37,8 +35,7 @@ translations. In this overview, we'll mainly discuss the core Zulip server and web application. -Usage assumptions and concepts ------------------------------- +## Usage assumptions and concepts Zulip is a real-time team chat application meant to provide a great experience for a wide range of organizations, from companies to @@ -50,7 +47,7 @@ all modern web browsers, several cross-protocol chat clients, and numerous dedicated [Zulip API](https://zulip.com/api) clients (e.g. bots). -A server can host multiple Zulip *realms* (organizations), each on its +A server can host multiple Zulip _realms_ (organizations), each on its own (sub)domain. While most installations host only one organization, some such as zulip.com host thousands. Each organization is a private chamber with its own users, streams, customizations, and so on. This @@ -61,10 +58,9 @@ more on security considerations and options, see [the security model section](../production/security-model.md) and the [Zulip Help Center](https://zulip.com/help). -Components ----------- +## Components - ![architecture-simple](../images/architecture_simple.png) +![architecture-simple](../images/architecture_simple.png) ### Django and Tornado @@ -116,8 +112,8 @@ For more details on the frontend, see our documentation on [directory structure](../overview/directory-structure.md), and [the static asset pipeline](../subsystems/html-css.html#static-asset-pipeline). -[Jinja2]: http://jinja.pocoo.org/ -[Handlebars]: https://handlebarsjs.com/ +[jinja2]: http://jinja.pocoo.org/ +[handlebars]: https://handlebarsjs.com/ ### nginx @@ -130,16 +126,16 @@ according to the rules laid down in the many config files found in important of these files. It explains what happens when requests come in from outside. -- In production, all requests to URLs beginning with `/static/` are - served from the corresponding files in `/home/zulip/prod-static/`, - and the production build process (`tools/build-release-tarball`) - compiles, minifies, and installs the static assets into the - `prod-static/` tree form. In development, files are served directly - from `/static/` in the Git repository. -- Requests to `/json/events` and `/api/v1/events`, i.e. the - real-time push system, are sent to the Tornado server. -- Requests to all other paths are sent to the Django app running via - `uWSGI` via `unix:/home/zulip/deployments/uwsgi-socket`. +- In production, all requests to URLs beginning with `/static/` are + served from the corresponding files in `/home/zulip/prod-static/`, + and the production build process (`tools/build-release-tarball`) + compiles, minifies, and installs the static assets into the + `prod-static/` tree form. In development, files are served directly + from `/static/` in the Git repository. +- Requests to `/json/events` and `/api/v1/events`, i.e. the + real-time push system, are sent to the Tornado server. +- Requests to all other paths are sent to the Django app running via + `uWSGI` via `unix:/home/zulip/deployments/uwsgi-socket`. - By default (i.e. if `LOCAL_UPLOADS_DIR` is set), nginx will serve user-uploaded content like avatars, custom emoji, and uploaded files. However, one can configure Zulip to store these in a cloud @@ -272,48 +268,48 @@ minimize the set of terminology listed here by giving elements self-explanatory names. - **bankruptcy**: When a user has been off Zulip for several days and - has hundreds of unread messages, they are prompted for whether - they want to mark all their unread messages as read. This is - called "declaring bankruptcy" (in reference to the concept in - finance). + has hundreds of unread messages, they are prompted for whether + they want to mark all their unread messages as read. This is + called "declaring bankruptcy" (in reference to the concept in + finance). - **chevron**: A small downward-facing arrow next to a message's - timestamp, offering contextual options, e.g., "Reply", "Mute [this - topic]", or "Link to this conversation". To avoid visual clutter, - the chevron only appears in the web UI upon hover. + timestamp, offering contextual options, e.g., "Reply", "Mute [this + topic]", or "Link to this conversation". To avoid visual clutter, + the chevron only appears in the web UI upon hover. - **ellipsis**: A small vertical three dot icon (technically called - as ellipsis-v), present in sidebars as a menu icon. - It offers contextual options for global filters (All messages - and Starred messages), stream filters and topics in left - sidebar and users in right sidebar. To avoid visual clutter - ellipsis only appears in the web UI upon hover. + as ellipsis-v), present in sidebars as a menu icon. + It offers contextual options for global filters (All messages + and Starred messages), stream filters and topics in left + sidebar and users in right sidebar. To avoid visual clutter + ellipsis only appears in the web UI upon hover. - **huddle**: What the codebase calls a "group private message". - **message editing**: If the realm admin allows it, then after a user - posts a message, the user has a few minutes to click "Edit" and - change the content of their message. If they do, Zulip adds a - marker such as "(EDITED)" at the top of the message, visible to - anyone who can see the message. + posts a message, the user has a few minutes to click "Edit" and + change the content of their message. If they do, Zulip adds a + marker such as "(EDITED)" at the top of the message, visible to + anyone who can see the message. - **realm**: What the codebase calls an "organization" in the UI. - **recipient bar**: A visual indication of the context of a message - or group of messages, displaying the stream and topic or private - message recipient list, at the top of a group of messages. A - typical 1-line message to a new recipient shows to the user as - three lines of content: first the recipient bar, second the - sender's name and avatar alongside the timestamp (and, on hover, - the star and the chevron), and third the message content. The - recipient bar is or contains hyperlinks to help the user narrow. + or group of messages, displaying the stream and topic or private + message recipient list, at the top of a group of messages. A + typical 1-line message to a new recipient shows to the user as + three lines of content: first the recipient bar, second the + sender's name and avatar alongside the timestamp (and, on hover, + the star and the chevron), and third the message content. The + recipient bar is or contains hyperlinks to help the user narrow. - **star**: Zulip allows a user to mark any message they can see, - public or private, as "starred". A user can easily access messages - they've starred through the "Starred messages" link in the - left sidebar, or use "is:starred" as a narrow or a search - constraint. Whether a user has or has not starred a particular - message is private; other users and realm admins don't know - whether a message has been starred, or by whom. + public or private, as "starred". A user can easily access messages + they've starred through the "Starred messages" link in the + left sidebar, or use "is:starred" as a narrow or a search + constraint. Whether a user has or has not starred a particular + message is private; other users and realm admins don't know + whether a message has been starred, or by whom. - **subject**: What the codebase calls a "topic" in many places. diff --git a/docs/overview/changelog.md b/docs/overview/changelog.md index 2ab3bacb1b..c988491d9d 100644 --- a/docs/overview/changelog.md +++ b/docs/overview/changelog.md @@ -749,26 +749,26 @@ details. #### Upgrade notes for 2.1.0 - The defaults for Zulip's now beta inline URL preview setting have changed. -Previously, the server-level `INLINE_URL_EMBED_PREVIEW` setting was -disabled, and organization-level setting was enabled. Now, the -server-level setting is enabled by default, and the organization-level -setting is disabled. As a result, organization administrators can -configure this feature entirely in the UI. However, servers that had -previously [enabled previews of linked -websites](https://zulip.com/help/allow-image-link-previews) will -lose the setting and need to re-enable it. + Previously, the server-level `INLINE_URL_EMBED_PREVIEW` setting was + disabled, and organization-level setting was enabled. Now, the + server-level setting is enabled by default, and the organization-level + setting is disabled. As a result, organization administrators can + configure this feature entirely in the UI. However, servers that had + previously [enabled previews of linked + websites](https://zulip.com/help/allow-image-link-previews) will + lose the setting and need to re-enable it. - We rewrote the Google authentication backend to use the `python-social-auth` system we use for other third-party authentication systems. For this release, the old variable names still work, but users should update the following setting names in their configuration as we will desupport the old names in a future release: - - In `/etc/zulip/zulip-secrets.conf`, `google_oauth2_client_secret` - is now called with `social_auth_google_secret`. - - In `/etc/zulip/settings.py`, `GOOGLE_OAUTH2_CLIENT_ID` should be - replaced with `SOCIAL_AUTH_GOOGLE_KEY`. - - In `/etc/zulip/settings.py`, `GoogleMobileOauth2Backend` should - be replaced with called `GoogleAuthBackend`. + - In `/etc/zulip/zulip-secrets.conf`, `google_oauth2_client_secret` + is now called with `social_auth_google_secret`. + - In `/etc/zulip/settings.py`, `GOOGLE_OAUTH2_CLIENT_ID` should be + replaced with `SOCIAL_AUTH_GOOGLE_KEY`. + - In `/etc/zulip/settings.py`, `GoogleMobileOauth2Backend` should + be replaced with called `GoogleAuthBackend`. - Installations using Zulip's LDAP integration without `LDAP_APPEND_DOMAIN` will need to configure two new settings telling Zulip how to look up a user in LDAP given their email address: @@ -807,6 +807,7 @@ lose the setting and need to re-enable it. downtime, and then upgrade to the new release. #### Full feature changelog + - Added sortable columns to all tables in settings pages. - Added webapp support for self-service public data exports. - Added 'e' keyboard shortcut for editing currently selected message. @@ -1014,6 +1015,7 @@ lose the setting and need to re-enable it. ### 2.0.0 -- 2019-03-01 #### Highlights + - Added automation for synchronizing user avatars, custom profile fields, disabled status, and more from LDAP/active directory. - Added support for explicitly setting oneself as "away" and "user @@ -1035,14 +1037,15 @@ lose the setting and need to re-enable it. #### Upgrade notes for 2.0.0 - This release adds support for submitting basic usage statistics to -help the Zulip core team. This feature can be enabled only if a server -is using the [Mobile Push Notification Service][mpns-statistics-docs], -and is enabled by default in that case. To disable it, set -`SUBMIT_USAGE_STATISTICS = False` in `/etc/zulip/settings.py`. + help the Zulip core team. This feature can be enabled only if a server + is using the [Mobile Push Notification Service][mpns-statistics-docs], + and is enabled by default in that case. To disable it, set + `SUBMIT_USAGE_STATISTICS = False` in `/etc/zulip/settings.py`. [mpns-statistics-docs]: ../production/mobile-push-notifications.html#submitting-statistics #### Full feature changelog + - Added support for CentOS 7 in the development environment provisioning process. This is an important step towards production CentOS/RHEL 7 support. @@ -1190,6 +1193,7 @@ Zulip installations; it has minimal changes for existing servers. doing an apt upgrade first. #### Full feature changelog + - Added an organization setting for message deletion time limits. - Added an organization setting to control who can edit topics. - Added Ctrl+K keyboard shortcut for getting to search (same as /, but @@ -1285,6 +1289,7 @@ Zulip installations; it has minimal changes for existing servers. ### 1.8.0 -- 2018-04-17 #### Highlights + - Dramatically simplified the server installation process; it's now possible to install Zulip without first setting up outgoing email. - Added experimental support for importing an organization's history @@ -1294,8 +1299,8 @@ Zulip installations; it has minimal changes for existing servers. - Lots of visual polish improvements. - Countless small bugfixes both in the backend and the UI. - **Security and privacy:** + - Several important security fixes since 1.7.0, which were released already in 1.7.1 and 1.7.2. - The security model for private streams has changed. Now @@ -1312,8 +1317,8 @@ Zulip installations; it has minimal changes for existing servers. - Added a user setting to control whether email notifications include message content (or just the fact that there are new messages). - **Visual and UI:** + - Added a user setting to translate emoticons/smileys to emoji. - Added a user setting to choose the emoji set used in Zulip: Google, Twitter, Apple, or Emoji One. @@ -1329,8 +1334,8 @@ Zulip installations; it has minimal changes for existing servers. - Improved the descriptions and UI for many settings. - Improved visual design of the help center (/help/). - **Core chat experience:** + - Added support for mentioning groups of users. - Added a setting to allow users to delete their messages. - Added support for uploading files in the message-edit UI. @@ -1390,8 +1395,8 @@ Zulip installations; it has minimal changes for existing servers. included in email subject lines. - Fixed uploading user avatars encoded using the CMYK mode. - **User accounts and invites:** + - Added support for users in multiple realms having the same email. - Added a display for whether the user is logged-in in logged-out pages. @@ -1406,8 +1411,8 @@ Zulip installations; it has minimal changes for existing servers. - Split the Notifications Stream setting in two settings, one for new users, the other for new streams. - **Stream subscriptions and settings:** + - Added traffic statistics (messages/week) to the "Manage streams" UI. - Fixed numerous issues in the "stream settings" UI. - Fixed numerous subtle bugs with the stream creation UI. @@ -1415,8 +1420,8 @@ Zulip installations; it has minimal changes for existing servers. so that they can be robust to streams being renamed. The change is backwards-compatible; existing narrow URLs still work. - **API, bots, and integrations:** + - Rewrote our API documentation to be much more friendly and expansive; it now covers most important endpoints, with nice examples. - New integrations: ErrBot, GoCD, Google Code-In, Opbeat, Groove, @@ -1434,8 +1439,8 @@ Zulip installations; it has minimal changes for existing servers. numbered-list format. - APIs for fetching messages now provide more metadata to help clients. - **Keyboard shortcuts:** + - Added new "basics" section to keyboard shortcuts documentation. - Added a new ">" keyboard shortcut for quote-and-reply. - Added a new "p" keyboard shortcut to jump to next unread PM thread. @@ -1443,16 +1448,16 @@ Zulip installations; it has minimal changes for existing servers. - Changed the hotkey for compose-private-message from "C" to "x". - Improve keyboard navigation of left and right sidebars with arrow keys. - **Mobile apps backend:** + - Added support for logging into the mobile apps with RemoteUserBackend. - Improved mobile notifications to support narrowing when one clicks a mobile push notification. - Statistics on the fraction of strings that are translated now include strings in the mobile apps as well. - **For server admins:** + - Added certbot support to the installer for getting certificates. - Added support for hosting multiple domains, not all as subdomains of the same base domain. @@ -1465,8 +1470,8 @@ Zulip installations; it has minimal changes for existing servers. - Improved Tornado retry logic for connecting to RabbitMQ. - Added a server setting to control whether digest emails are sent. - **For Zulip developers:** + - Migrated the codebase to use the nice Python 3 typing syntax. - Added a new /team/ page explaining the team, with a nice visualization of our contributors. @@ -1708,28 +1713,28 @@ running a version from before 1.7 should upgrade directly to 1.7.1. #### Highlights - A complete visual redesign of the logged-out pages, including login, -registration, integrations, etc. + registration, integrations, etc. - New visual designs for numerous UI elements, including the emoji -picker, user profile popovers, sidebars, compose, and many more. + picker, user profile popovers, sidebars, compose, and many more. - A complete redesign of the Zulip settings interfaces to look a lot -nicer and be easier to navigate. + nicer and be easier to navigate. - Organization admins can now configure the login and registration -pages to show visitors a nice organization profile with custom text -and images, written in Markdown. + pages to show visitors a nice organization profile with custom text + and images, written in Markdown. - Massively improved performance for presence and settings pages, -especially for very large organizations (1000+ users). + especially for very large organizations (1000+ users). - A dozen useful new keyboard shortcuts, from editing messages to -emoji reactions to drafts and managing streams. + emoji reactions to drafts and managing streams. - Typing notifications for private message threads. - Users can now change their own email address. - New saved-drafts feature. - The server can now run on a machine with as little as 2GB of RAM. - The new [Electron desktop app][electron-app] and new -[React Native mobile app for iOS][ios-app] are now the recommended -Zulip apps. + [React Native mobile app for iOS][ios-app] are now the recommended + Zulip apps. - Mobile web now works much better, especially on iOS. - Support for sending mobile push notifications via -[a new forwarding service][mobile-push] + [a new forwarding service][mobile-push] - Complete translations for Spanish, German, and Czech (and expanded partial translations for Japanese, Chinese, French, Hungarian, Polish, Dutch, Russian, Bulgarian, Portuguese, @@ -2068,6 +2073,7 @@ Zulip apps. open source. ### 1.3.13 - 2016-06-21 + - Added nearly complete internationalization of the Zulip UI. - Added warning when using @all/@everyone. - Added button offering to subscribe at bottom of narrows to streams @@ -2105,6 +2111,7 @@ Zulip apps. - Removed most of the remaining JavaScript global variables. ### 1.3.12 - 2016-05-10 + - CVE-2016-4426: Bot API keys were accessible to other users in the same realm. - CVE-2016-4427: Deactivated users could access messages if SSO was enabled. - Fixed a RabbitMQ configuration bug that resulted in reordered messages. @@ -2112,6 +2119,7 @@ Zulip apps. - Added an option to logout_all_users to delete only sessions for deactivated users. ### 1.3.11 - 2016-05-02 + - Moved email digest support into the default Zulip production configuration. - Added options for configuring PostgreSQL, RabbitMQ, Redis, and memcached in settings.py. @@ -2140,6 +2148,7 @@ Zulip apps. - Fixed Jira integration incorrectly not @-mentioning assignee. ### 1.3.10 - 2016-01-21 + - Added new integration for Travis CI. - Added settings option to control maximum file upload size. - Added support for running Zulip development environment in Docker. @@ -2162,9 +2171,11 @@ Zulip apps. - Substantially cleaned up console logging from run-dev.py. ### 1.3.9 - 2015-11-16 + - Fixed buggy #! lines in upgrade scripts. ### 1.3.8 - 2015-11-15 + - Added options to the Python API for working with untrusted server certificates. - Added a lot of documentation on the development environment and testing. - Added partial support for translating the Zulip UI. @@ -2180,6 +2191,7 @@ Zulip apps. - Major preliminary progress towards supporting Python 3. ### 1.3.7 - 2015-10-19 + - Turn off desktop and audible notifications for streams by default. - Added support for the LDAP authentication integration creating new users. - Added new endpoint to support Google auth on mobile. diff --git a/docs/overview/directory-structure.md b/docs/overview/directory-structure.md index 611293ff28..31eaebd434 100644 --- a/docs/overview/directory-structure.md +++ b/docs/overview/directory-structure.md @@ -30,8 +30,7 @@ paths will be familiar to Django developers. - `zerver/views/*.py` Most [Django views](https://docs.djangoproject.com/en/1.8/topics/http/views/). -- `zerver/webhooks/` Webhook views and tests for [Zulip's incoming webhook integrations]( - https://zulip.com/api/incoming-webhooks-overview). +- `zerver/webhooks/` Webhook views and tests for [Zulip's incoming webhook integrations](https://zulip.com/api/incoming-webhooks-overview). - `zerver/tornado/views.py` Tornado views. @@ -41,7 +40,7 @@ paths will be familiar to Django developers. - `zproject/backends.py` [Authentication backends](https://docs.djangoproject.com/en/1.8/topics/auth/customizing/). -------------------------------------------------------------------- +--- ### HTML templates @@ -53,7 +52,7 @@ templating systems. - `static/templates/` [Handlebars](https://handlebarsjs.com/) templates for the frontend. ----------------------------------------- +--- ### JavaScript, TypeScript, and other static assets @@ -68,9 +67,9 @@ templating systems. - `node_modules/` Third-party JavaScript installed via `yarn`. - `static/assets/` For assets not to be served to the web (e.g. the system to - generate our favicons). + generate our favicons). ------------------------------------------------------------------------ +--- ### Tests @@ -82,7 +81,7 @@ templating systems. - `tools/test-*` Developer-facing test runner scripts. ------------------------------------------------------ +--- ### Management commands @@ -95,10 +94,10 @@ Django context (i.e. with database access). deactivate a user properly). - `zilencer/management/commands/` includes some dev-specific - commands such as `populate_db`, which are not included in - the production distribution. + commands such as `populate_db`, which are not included in + the production distribution. ---------------------------------------------------------------- +--- ### Scripts @@ -124,7 +123,7 @@ Django context (i.e. with database access). set up and run our tests in CI. Actual test suites should go in `tools/`. ---------------------------------------------------------- +--- ### API and bots @@ -136,7 +135,7 @@ Django context (i.e. with database access). - `templates/zerver/integrations/` (within `templates/zerver/`, above). Documentation for these integrations. -------------------------------------------------------------------------- +--- ### Production Puppet configuration @@ -146,7 +145,7 @@ This is used to deploy essentially all configuration in production. - `puppet/zulip/manifests/profile/standalone.pp` Main manifest for Zulip standalone deployments. ------------------------------------------------------------------------ +--- ### Additional Django apps @@ -161,25 +160,25 @@ This is used to deploy essentially all configuration in production. - `zilencer` Primarily used to hold management commands that aren't used in production. Not included in production distribution. ------------------------------------------------------------------------ +--- ### Jinja2 compatibility files - `zproject/jinja2/__init__.py` Jinja2 environment. ------------------------------------------------------------------------ +--- ### Translation files - `locale/` Backend (Django) and frontend translation data files. ------------------------------------------------------------------------ +--- ### Documentation -- `docs/` Source for this documentation. +- `docs/` Source for this documentation. --------------------------------------------------------------- +--- You can consult the repository's `.gitattributes` file to see exactly which components are excluded from production releases (release diff --git a/docs/overview/release-lifecycle.md b/docs/overview/release-lifecycle.md index b9dbb12266..14228d1412 100644 --- a/docs/overview/release-lifecycle.md +++ b/docs/overview/release-lifecycle.md @@ -232,14 +232,8 @@ independently as needed. [chat-zulip-org]: ../contributing/chat-zulip-org.md [fork-zulip]: ../production/upgrade-or-modify.html#modifying-zulip [zulip-server]: https://github.com/zulip/zulip - [mobile-beta]: https://github.com/zulip/zulip-mobile#using-the-beta - -[label-blocker]: -https://github.com/zulip/zulip/issues?q=is%3Aissue+is%3Aopen+label%3A%22priority%3A+blocker%22 -[label-high]: -https://github.com/zulip/zulip/issues?q=is%3Aissue+is%3Aopen+label%3A%22priority%3A+high%22 -[label-release-goal]: -https://github.com/zulip/zulip/issues?q=is%3Aissue+is%3Aopen+label%3A%22release+goal%22 -[label-post-release]: -https://github.com/zulip/zulip/issues?q=is%3Aissue+is%3Aopen+label%3A%22post+release%22 +[label-blocker]: https://github.com/zulip/zulip/issues?q=is%3Aissue+is%3Aopen+label%3A%22priority%3A+blocker%22 +[label-high]: https://github.com/zulip/zulip/issues?q=is%3Aissue+is%3Aopen+label%3A%22priority%3A+high%22 +[label-release-goal]: https://github.com/zulip/zulip/issues?q=is%3Aissue+is%3Aopen+label%3A%22release+goal%22 +[label-post-release]: https://github.com/zulip/zulip/issues?q=is%3Aissue+is%3Aopen+label%3A%22post+release%22 diff --git a/docs/production/authentication-methods.md b/docs/production/authentication-methods.md index 382a37de69..fb98db1a7a 100644 --- a/docs/production/authentication-methods.md +++ b/docs/production/authentication-methods.md @@ -26,6 +26,7 @@ creating the initial realm and user. You can disable it after that. With just a few lines of configuration, your Zulip server can authenticate users with any of several single-sign-on (SSO) authentication providers: + - Google accounts, with `GoogleAuthBackend` - GitHub accounts, with `GitHubAuthBackend` - GitLab accounts, with `GitLabAuthBackend` @@ -38,6 +39,7 @@ are documented in your `settings.py`. ```eval_rst .. _ldap: ``` + ## LDAP (including Active Directory) Zulip supports retrieving information about users via LDAP, and @@ -51,6 +53,7 @@ In either configuration, you will need to do the following: organization using LDAP authentication. 1. Tell Zulip how to connect to your LDAP server: + - Fill out the section of your `/etc/zulip/settings.py` headed "LDAP integration, part 1: Connecting to the LDAP server". - If a password is required, put it in @@ -61,6 +64,7 @@ In either configuration, you will need to do the following: 1. Decide how you want to map the information in your LDAP database to users' account data in Zulip. For each Zulip user, two closely related concepts are: + - their **email address**. Zulip needs this in order to send, for example, a notification when they're offline and another user sends a PM. @@ -78,32 +82,35 @@ In either configuration, you will need to do the following: ways to set up the username and/or email mapping: (A) Using email addresses as Zulip usernames, if LDAP has each - user's email address: - - Make `AUTH_LDAP_USER_SEARCH` a query by email address. - - Set `AUTH_LDAP_REVERSE_EMAIL_SEARCH` to the same query with - `%(email)s` rather than `%(user)s` as the search parameter. - - Set `AUTH_LDAP_USERNAME_ATTR` to the name of the LDAP - attribute for the user's LDAP username in the search result - for `AUTH_LDAP_REVERSE_EMAIL_SEARCH`. + user's email address: + + - Make `AUTH_LDAP_USER_SEARCH` a query by email address. + - Set `AUTH_LDAP_REVERSE_EMAIL_SEARCH` to the same query with + `%(email)s` rather than `%(user)s` as the search parameter. + - Set `AUTH_LDAP_USERNAME_ATTR` to the name of the LDAP + attribute for the user's LDAP username in the search result + for `AUTH_LDAP_REVERSE_EMAIL_SEARCH`. (B) Using LDAP usernames as Zulip usernames, with email addresses - formed consistently like `sam` -> `sam@example.com`: - - Set `AUTH_LDAP_USER_SEARCH` to query by LDAP username - - Set `LDAP_APPEND_DOMAIN = "example.com"`. + formed consistently like `sam` -> `sam@example.com`: + + - Set `AUTH_LDAP_USER_SEARCH` to query by LDAP username + - Set `LDAP_APPEND_DOMAIN = "example.com"`. (C) Using LDAP usernames as Zulip usernames, with email addresses - taken from some other attribute in LDAP (for example, `mail`): - - Set `AUTH_LDAP_USER_SEARCH` to query by LDAP username - - Set `LDAP_EMAIL_ATTR = "mail"`. - - Set `AUTH_LDAP_REVERSE_EMAIL_SEARCH` to a query that will find - an LDAP user given their email address (i.e. a search by - `LDAP_EMAIL_ATTR`). For example: - ```python - AUTH_LDAP_REVERSE_EMAIL_SEARCH = LDAPSearch("ou=users,dc=example,dc=com", - ldap.SCOPE_SUBTREE, "(mail=%(email)s)") - ``` - - Set `AUTH_LDAP_USERNAME_ATTR` to the name of the LDAP - attribute for the user's LDAP username in that search result. + taken from some other attribute in LDAP (for example, `mail`): + + - Set `AUTH_LDAP_USER_SEARCH` to query by LDAP username + - Set `LDAP_EMAIL_ATTR = "mail"`. + - Set `AUTH_LDAP_REVERSE_EMAIL_SEARCH` to a query that will find + an LDAP user given their email address (i.e. a search by + `LDAP_EMAIL_ATTR`). For example: + ```python + AUTH_LDAP_REVERSE_EMAIL_SEARCH = LDAPSearch("ou=users,dc=example,dc=com", + ldap.SCOPE_SUBTREE, "(mail=%(email)s)") + ``` + - Set `AUTH_LDAP_USERNAME_ATTR` to the name of the LDAP + attribute for the user's LDAP username in that search result. You can quickly test whether your configuration works by running: @@ -119,22 +126,23 @@ email address, if it isn't the same as the "Zulip username"). of the following configurations: - To access by Active Directory username: - ```python - AUTH_LDAP_USER_SEARCH = LDAPSearch("ou=users,dc=example,dc=com", - ldap.SCOPE_SUBTREE, "(sAMAccountName=%(user)s)") - AUTH_LDAP_REVERSE_EMAIL_SEARCH = LDAPSearch("ou=users,dc=example,dc=com", - ldap.SCOPE_SUBTREE, "(mail=%(email)s)") - AUTH_LDAP_USERNAME_ATTR = "sAMAccountName" - ``` + + ```python + AUTH_LDAP_USER_SEARCH = LDAPSearch("ou=users,dc=example,dc=com", + ldap.SCOPE_SUBTREE, "(sAMAccountName=%(user)s)") + AUTH_LDAP_REVERSE_EMAIL_SEARCH = LDAPSearch("ou=users,dc=example,dc=com", + ldap.SCOPE_SUBTREE, "(mail=%(email)s)") + AUTH_LDAP_USERNAME_ATTR = "sAMAccountName" + ``` - To access by Active Directory email address: - ```python - AUTH_LDAP_USER_SEARCH = LDAPSearch("ou=users,dc=example,dc=com", - ldap.SCOPE_SUBTREE, "(mail=%(user)s)") - AUTH_LDAP_REVERSE_EMAIL_SEARCH = LDAPSearch("ou=users,dc=example,dc=com", - ldap.SCOPE_SUBTREE, "(mail=%(email)s)") - AUTH_LDAP_USERNAME_ATTR = "mail" - ``` + ```python + AUTH_LDAP_USER_SEARCH = LDAPSearch("ou=users,dc=example,dc=com", + ldap.SCOPE_SUBTREE, "(mail=%(user)s)") + AUTH_LDAP_REVERSE_EMAIL_SEARCH = LDAPSearch("ou=users,dc=example,dc=com", + ldap.SCOPE_SUBTREE, "(mail=%(email)s)") + AUTH_LDAP_USERNAME_ATTR = "mail" + ``` **If you are using LDAP for authentication**: you will need to enable the `zproject.backends.ZulipLDAPAuthBackend` auth backend, in @@ -168,6 +176,7 @@ We recommend running this command in a **regular cron job**, to pick up changes made on your LDAP server. All of these data synchronization options have the same model: + - New users will be populated automatically with the name/avatar/etc. from LDAP (as configured) on account creation. - The `manage.py sync_ldap_user_data` cron job will automatically @@ -270,6 +279,7 @@ the fields that would be useful to sync from your LDAP databases. ### Multiple LDAP searches To do the union of multiple LDAP searches, use `LDAPSearchUnion`. For example: + ```python AUTH_LDAP_USER_SEARCH = LDAPSearchUnion( LDAPSearch("ou=users,dc=example,dc=com", ldap.SCOPE_SUBTREE, "(uid=%(user)s)"), @@ -300,6 +310,7 @@ For the root subdomain, `www` in the list will work, or any other of For example, with `org_membership` set to `department`, a user with the following attributes will have access to the root and `engineering` subdomains: + ```text ... department: engineering @@ -364,26 +375,27 @@ it as follows: 1. Tell your IdP how to find your Zulip server: - - **SP Entity ID**: `https://yourzulipdomain.example.com`. + - **SP Entity ID**: `https://yourzulipdomain.example.com`. - The `Entity ID` should match the value of - `SOCIAL_AUTH_SAML_SP_ENTITY_ID` computed in the Zulip settings. - You can get the correct value by running the following: - `/home/zulip/deployments/current/scripts/get-django-setting SOCIAL_AUTH_SAML_SP_ENTITY_ID`. + The `Entity ID` should match the value of + `SOCIAL_AUTH_SAML_SP_ENTITY_ID` computed in the Zulip settings. + You can get the correct value by running the following: + `/home/zulip/deployments/current/scripts/get-django-setting SOCIAL_AUTH_SAML_SP_ENTITY_ID`. - - **SSO URL**: - `https://yourzulipdomain.example.com/complete/saml/`. This is - the "SAML ACS url" in SAML terminology. + - **SSO URL**: + `https://yourzulipdomain.example.com/complete/saml/`. This is + the "SAML ACS url" in SAML terminology. - If you're - [hosting multiple organizations](../production/multiple-organizations.html#authentication), - you need to use `SOCIAL_AUTH_SUBDOMAIN`. For example, - if `SOCIAL_AUTH_SUBDOMAIN="auth"` and `EXTERNAL_HOST=zulip.example.com`, - this should be `https://auth.zulip.example.com/complete/saml/`. + If you're + [hosting multiple organizations](../production/multiple-organizations.html#authentication), + you need to use `SOCIAL_AUTH_SUBDOMAIN`. For example, + if `SOCIAL_AUTH_SUBDOMAIN="auth"` and `EXTERNAL_HOST=zulip.example.com`, + this should be `https://auth.zulip.example.com/complete/saml/`. -2. Tell Zulip how to connect to your SAML provider(s) by filling +1. Tell Zulip how to connect to your SAML provider(s) by filling out the section of `/etc/zulip/settings.py` on your Zulip server with the heading "SAML Authentication". + - You will need to update `SOCIAL_AUTH_SAML_ORG_INFO` with your organization name (`displayname` may appear in the IdP's authentication flow; `name` won't be displayed to humans). @@ -411,29 +423,29 @@ it as follows: 6. The `auto_signup` field determines how Zulip should handle login attempts by users who don't have an account yet. -3. Install the certificate(s) required for SAML authentication. You - will definitely need the public certificate of your IdP. Some IdP - providers also support the Zulip server (Service Provider) having - a certificate used for encryption and signing. We detail these - steps as optional below, because they aren't required for basic - setup, and some IdPs like Okta don't fully support Service - Provider certificates. You should install them as follows: +1. Install the certificate(s) required for SAML authentication. You + will definitely need the public certificate of your IdP. Some IdP + providers also support the Zulip server (Service Provider) having + a certificate used for encryption and signing. We detail these + steps as optional below, because they aren't required for basic + setup, and some IdPs like Okta don't fully support Service + Provider certificates. You should install them as follows: - 1. On your Zulip server, `mkdir -p /etc/zulip/saml/idps/` - 2. Put the IDP public certificate in `/etc/zulip/saml/idps/{idp_name}.crt` - 3. (Optional) Put the Zulip server public certificate in `/etc/zulip/saml/zulip-cert.crt` - and the corresponding private key in `/etc/zulip/saml/zulip-private-key.key`. Note that - the certificate should be the single X.509 certificate for the server, not a full chain of - trust, which consists of multiple certificates. - 4. Set the proper permissions on these files and directories: + 1. On your Zulip server, `mkdir -p /etc/zulip/saml/idps/` + 2. Put the IDP public certificate in `/etc/zulip/saml/idps/{idp_name}.crt` + 3. (Optional) Put the Zulip server public certificate in `/etc/zulip/saml/zulip-cert.crt` + and the corresponding private key in `/etc/zulip/saml/zulip-private-key.key`. Note that + the certificate should be the single X.509 certificate for the server, not a full chain of + trust, which consists of multiple certificates. + 4. Set the proper permissions on these files and directories: - ```bash - chown -R zulip.zulip /etc/zulip/saml/ - find /etc/zulip/saml/ -type f -exec chmod 644 -- {} + - chmod 640 /etc/zulip/saml/zulip-private-key.key - ``` + ```bash + chown -R zulip.zulip /etc/zulip/saml/ + find /etc/zulip/saml/ -type f -exec chmod 644 -- {} + + chmod 640 /etc/zulip/saml/zulip-private-key.key + ``` -4. (Optional) If you configured the optional public and private server +1. (Optional) If you configured the optional public and private server certificates above, you can enable the additional setting `"authnRequestsSigned": True` in `SOCIAL_AUTH_SAML_SECURITY_CONFIG` to have the SAMLRequests the server will be issuing to the IdP @@ -442,18 +454,18 @@ it as follows: assertions in the SAMLResponses the IdP will send about authenticated users. -5. Enable the `zproject.backends.SAMLAuthBackend` auth backend, in -`AUTHENTICATION_BACKENDS` in `/etc/zulip/settings.py`. +1. Enable the `zproject.backends.SAMLAuthBackend` auth backend, in + `AUTHENTICATION_BACKENDS` in `/etc/zulip/settings.py`. -6. [Restart the Zulip server](../production/settings.md) to ensure -your settings changes take effect. The Zulip login page should now -have a button for SAML authentication that you can use to log in or -create an account (including when creating a new organization). +1. [Restart the Zulip server](../production/settings.md) to ensure + your settings changes take effect. The Zulip login page should now + have a button for SAML authentication that you can use to log in or + create an account (including when creating a new organization). -7. If the configuration was successful, the server's metadata can be -found at `https://yourzulipdomain.example.com/saml/metadata.xml`. You -can use this for verifying your configuration or provide it to your -IdP. +1. If the configuration was successful, the server's metadata can be + found at `https://yourzulipdomain.example.com/saml/metadata.xml`. You + can use this for verifying your configuration or provide it to your + IdP. [saml-help-center]: https://zulip.com/help/saml-authentication @@ -502,7 +514,6 @@ to the root and `engineering` subdomains: ``` - ## Apache-based SSO with `REMOTE_USER` If you have any existing SSO solution where a preferred way to deploy @@ -542,6 +553,7 @@ straightforward way to deploy that SSO solution with Zulip. using the `htpasswd` example configuration and demonstrate that working end-to-end, before returning later to configure your SSO solution. You can do that with the following steps: + ```bash /home/zulip/deployments/current/scripts/restart-server cd /etc/apache2/sites-available/ @@ -578,11 +590,11 @@ improve this SSO setup documentation are very welcome! - The following log files can be helpful when debugging this setup: - - `/var/log/zulip/{errors.log,server.log}` (the usual places) - - `/var/log/nginx/access.log` (nginx access logs) - - `/var/log/apache2/zulip_auth_access.log` (from the - `zulip-sso.conf` Apache config file; you may want to change - `LogLevel` in that file to "debug" to make this more verbose) + - `/var/log/zulip/{errors.log,server.log}` (the usual places) + - `/var/log/nginx/access.log` (nginx access logs) + - `/var/log/apache2/zulip_auth_access.log` (from the + `zulip-sso.conf` Apache config file; you may want to change + `LogLevel` in that file to "debug" to make this more verbose) ### Life of an Apache-based SSO login attempt @@ -594,7 +606,7 @@ to debug. - Since you've configured `/etc/zulip/settings.py` to only define the `zproject.backends.ZulipRemoteUserBackend`, `zproject/computed_settings.py` configures `/accounts/login/sso/` as - `HOME_NOT_LOGGED_IN`. This makes `https://zulip.example.com/` + `HOME_NOT_LOGGED_IN`. This makes `https://zulip.example.com/` (a.k.a. the homepage for the main Zulip Django app running behind nginx) redirect to `/accounts/login/sso/` for a user that isn't logged in. @@ -626,9 +638,9 @@ Zulip supports using the web flow for Sign in with Apple on self-hosted servers. To do so, you'll need to do the following: 1. Visit [the Apple Developer site][apple-developer] and [Create a -Services ID.][apple-create-services-id]. When prompted for a "Return -URL", enter `https://zulip.example.com/complete/apple/` (using the -domain for your server). + Services ID.][apple-create-services-id]. When prompted for a "Return + URL", enter `https://zulip.example.com/complete/apple/` (using the + domain for your server). 1. Create a [Sign in with Apple private key][apple-create-private-key]. @@ -642,6 +654,7 @@ domain for your server). ``` 1. Configure Apple authentication in `/etc/zulip/settings.py`: + - `SOCIAL_AUTH_APPLE_TEAM`: Your Team ID from Apple, which is a string like "A1B2C3D4E5". - `SOCIAL_AUTH_APPLE_SERVICES_ID`: The Services ID you created in diff --git a/docs/production/deployment.md b/docs/production/deployment.md index f7ea66b51f..95d19ab3e3 100644 --- a/docs/production/deployment.md +++ b/docs/production/deployment.md @@ -191,15 +191,15 @@ configure that as follows: with `/home/zulip/deployments/current/scripts/restart-server`. 1. Add the following block to `/etc/zulip/zulip.conf`: - ```ini - [application_server] - nginx_listen_port = 12345 - ``` + ```ini + [application_server] + nginx_listen_port = 12345 + ``` 1. As root, run - `/home/zulip/deployments/current/scripts/zulip-puppet-apply`. This - will convert Zulip's main `nginx` configuration file to use your new - port. + `/home/zulip/deployments/current/scripts/zulip-puppet-apply`. This + will convert Zulip's main `nginx` configuration file to use your new + port. We also have documentation for a Zulip server [using HTTP][using-http] for use behind reverse proxies. @@ -219,23 +219,24 @@ To use Smokescreen: 1. Add `, zulip::profile::smokescreen` to the list of `puppet_classes` in `/etc/zulip/zulip.conf`. A typical value after this change is: - ```ini - puppet_classes = zulip::profile::standalone, zulip::profile::smokescreen - ``` + + ```ini + puppet_classes = zulip::profile::standalone, zulip::profile::smokescreen + ``` 1. Optionally, configure the [smokescreen ACLs][smokescreen-acls]. By - default, Smokescreen denies access to all [non-public IP - addresses](https://en.wikipedia.org/wiki/Private_network), including - 127.0.0.1. + default, Smokescreen denies access to all [non-public IP + addresses](https://en.wikipedia.org/wiki/Private_network), including + 127.0.0.1. 1. Add the following block to `/etc/zulip/zulip.conf`, substituting in your proxy's hostname/IP and port: - ```ini - [http_proxy] - host = 127.0.0.1 - port = 4750 - ``` + ```ini + [http_proxy] + host = 127.0.0.1 + port = 4750 + ``` 1. If you intend to also make the Smokescreen install available to other hosts, set `listen_address` in the same block. Note that you @@ -280,22 +281,23 @@ HTTP as follows: 1. Add the following block to `/etc/zulip/zulip.conf`: - ```ini - [application_server] - http_only = true - ``` + ```ini + [application_server] + http_only = true + ``` 1. As root, run -`/home/zulip/deployments/current/scripts/zulip-puppet-apply`. This -will convert Zulip's main `nginx` configuration file to allow HTTP -instead of HTTPS. + `/home/zulip/deployments/current/scripts/zulip-puppet-apply`. This + will convert Zulip's main `nginx` configuration file to allow HTTP + instead of HTTPS. 1. Finally, restart the Zulip server, using -`/home/zulip/deployments/current/scripts/restart-server`. + `/home/zulip/deployments/current/scripts/restart-server`. ### nginx configuration For `nginx` configuration, there's two things you need to set up: + - The root `nginx.conf` file. We recommend using `/etc/nginx/nginx.conf` from your Zulip server for our recommended settings. E.g. if you don't set `client_max_body_size`, it won't be @@ -341,12 +343,12 @@ make the following changes in two configuration files. 1. Follow the instructions for [Configure Zulip to allow HTTP](#configuring-zulip-to-allow-http). 2. Add the following to `/etc/zulip/settings.py`: - ```python - EXTERNAL_HOST = 'zulip.example.com' - ALLOWED_HOSTS = ['zulip.example.com', '127.0.0.1'] - USE_X_FORWARDED_HOST = True - ``` + ```python + EXTERNAL_HOST = 'zulip.example.com' + ALLOWED_HOSTS = ['zulip.example.com', '127.0.0.1'] + USE_X_FORWARDED_HOST = True + ``` 3. Restart your Zulip server with `/home/zulip/deployments/current/scripts/restart-server`. @@ -357,41 +359,41 @@ make the following changes in two configuration files. and then run `a2ensite zulip.example.com && systemctl reload apache2`): - ```apache - - ServerName zulip.example.com - RewriteEngine On - RewriteRule ^ https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L] - + ```apache + + ServerName zulip.example.com + RewriteEngine On + RewriteRule ^ https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L] + - - ServerName zulip.example.com + + ServerName zulip.example.com - RequestHeader set "X-Forwarded-Proto" expr=%{REQUEST_SCHEME} - RequestHeader set "X-Forwarded-SSL" expr=%{HTTPS} + RequestHeader set "X-Forwarded-Proto" expr=%{REQUEST_SCHEME} + RequestHeader set "X-Forwarded-SSL" expr=%{HTTPS} - RewriteEngine On - RewriteRule /(.*) http://localhost:5080/$1 [P,L] + RewriteEngine On + RewriteRule /(.*) http://localhost:5080/$1 [P,L] - - Require all granted - ProxyPass http://localhost:5080/ timeout=300 - ProxyPassReverse http://localhost:5080/ - ProxyPassReverseCookieDomain 127.0.0.1 zulip.example.com - + + Require all granted + ProxyPass http://localhost:5080/ timeout=300 + ProxyPassReverse http://localhost:5080/ + ProxyPassReverseCookieDomain 127.0.0.1 zulip.example.com + - SSLEngine on - SSLProxyEngine on - SSLCertificateFile /etc/letsencrypt/live/zulip.example.com/fullchain.pem - SSLCertificateKeyFile /etc/letsencrypt/live/zulip.example.com/privkey.pem - SSLOpenSSLConfCmd DHParameters "/etc/nginx/dhparam.pem" - SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1 - SSLCipherSuite ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384 - SSLHonorCipherOrder off - SSLSessionTickets off - Header set Strict-Transport-Security "max-age=31536000" - - ``` + SSLEngine on + SSLProxyEngine on + SSLCertificateFile /etc/letsencrypt/live/zulip.example.com/fullchain.pem + SSLCertificateKeyFile /etc/letsencrypt/live/zulip.example.com/privkey.pem + SSLOpenSSLConfCmd DHParameters "/etc/nginx/dhparam.pem" + SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1 + SSLCipherSuite ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384 + SSLHonorCipherOrder off + SSLSessionTickets off + Header set Strict-Transport-Security "max-age=31536000" + + ``` ### HAProxy configuration @@ -416,16 +418,16 @@ If you're using another reverse proxy implementation, there are few things you need to be careful about when configuring it: 1. Configure your reverse proxy (or proxies) to correctly maintain the -`X-Forwarded-For` HTTP header, which is supposed to contain the series -of IP addresses the request was forwarded through. You can verify -your work by looking at `/var/log/zulip/server.log` and checking it -has the actual IP addresses of clients, not the IP address of the -proxy server. + `X-Forwarded-For` HTTP header, which is supposed to contain the series + of IP addresses the request was forwarded through. You can verify + your work by looking at `/var/log/zulip/server.log` and checking it + has the actual IP addresses of clients, not the IP address of the + proxy server. 2. Ensure your proxy doesn't interfere with Zulip's use of -long-polling for real-time push from the server to your users' -browsers. This [nginx code snippet][nginx-proxy-longpolling-config] -does this. + long-polling for real-time push from the server to your users' + browsers. This [nginx code snippet][nginx-proxy-longpolling-config] + does this. The key configuration options are, for the `/json/events` and `/api/1/events` endpoints: @@ -436,14 +438,13 @@ The key configuration options are, for the `/json/events` and return occasional 502 errors to clients using Zulip's events API. 3. The other tricky failure mode we've seen with `nginx` reverse -proxies is that they can load-balance between the IPv4 and IPv6 -addresses for a given hostname. This can result in mysterious errors -that can be quite difficult to debug. Be sure to declare your -`upstreams` equivalent in a way that won't do load-balancing -unexpectedly (e.g. pointing to a DNS name that you haven't configured -with multiple IPs for your Zulip machine; sometimes this happens with -IPv6 configuration). - + proxies is that they can load-balance between the IPv4 and IPv6 + addresses for a given hostname. This can result in mysterious errors + that can be quite difficult to debug. Be sure to declare your + `upstreams` equivalent in a way that won't do load-balancing + unexpectedly (e.g. pointing to a DNS name that you haven't configured + with multiple IPs for your Zulip machine; sometimes this happens with + IPv6 configuration). ## System and deployment configuration @@ -461,11 +462,12 @@ The most common is **`zulip::profile::standalone`**, used for a stand-alone single-host deployment. [Components](../overview/architecture-overview.html#components) of that include: - - **`zulip::profile::app_frontend`** - - **`zulip::profile::memcached`** - - **`zulip::profile::postgresql`** - - **`zulip::profile::redis`** - - **`zulip::profile::rabbitmq`** + +- **`zulip::profile::app_frontend`** +- **`zulip::profile::memcached`** +- **`zulip::profile::postgresql`** +- **`zulip::profile::redis`** +- **`zulip::profile::rabbitmq`** If you are using a [Apache as a single-sign-on authenticator](../production/authentication-methods.html#apache-based-sso-with-remote-user), @@ -477,8 +479,6 @@ Set to the string `enabled` if enabling the [multi-language PGroonga search extension](../subsystems/full-text-search.html#multi-language-full-text-search). - - ### `[deployment]` #### `deploy_options` @@ -486,12 +486,12 @@ extension](../subsystems/full-text-search.html#multi-language-full-text-search). Options passed by `upgrade-zulip` and `upgrade-zulip-from-git` into `upgrade-zulip-stage-2`. These might be any of: - - **`--skip-puppet`** skips doing Puppet/apt upgrades. The user will need - to run `zulip-puppet-apply` manually after the upgrade. - - **`--skip-migrations`** skips running database migrations. The - user will need to run `./manage.py migrate` manually after the upgrade. - - **`--skip-purge-old-deployments`** skips purging old deployments; - without it, only deployments with the last two weeks are kept. +- **`--skip-puppet`** skips doing Puppet/apt upgrades. The user will need + to run `zulip-puppet-apply` manually after the upgrade. +- **`--skip-migrations`** skips running database migrations. The + user will need to run `./manage.py migrate` manually after the upgrade. +- **`--skip-purge-old-deployments`** skips purging old deployments; + without it, only deployments with the last two weeks are kept. Generally installations will not want to set any of these options; the `--skip-*` options are primarily useful for reducing upgrade downtime @@ -502,8 +502,6 @@ for servers that are upgraded frequently by core Zulip developers. Default repository URL used when [upgrading from a Git repository](../production/upgrade-or-modify.html#upgrading-from-a-git-repository). - - ### `[application_server]` #### `http_only` @@ -553,8 +551,6 @@ Override the default uwsgi backlog of 128 connections. Override the default `uwsgi` (Django) process count of 6 on hosts with more than 3.5GiB of RAM, 4 on hosts with less. - - ### `[certbot]` #### `auto_renew` @@ -563,8 +559,6 @@ If set to the string `yes`, [Certbot will attempt to automatically renew its certificate](../production/ssl-certificates.html#certbot-recommended). Do no set by hand; use `scripts/setup/setup-certbot` to configure this. - - ### `[postfix]` #### `mailname` @@ -615,16 +609,12 @@ connections. The version of PostgreSQL that is in use. Do not set by hand; use the [PostgreSQL upgrade tool](../production/upgrade-or-modify.html#upgrading-postgresql). - - ### `[rabbitmq]` #### `nodename` The name used to identify the local RabbitMQ server; do not modify. - - ### `[memcached]` #### `memory` @@ -632,8 +622,6 @@ The name used to identify the local RabbitMQ server; do not modify. Override the number of megabytes of memory that memcached should be configured to consume; defaults to 1/8th of the total server memory. - - ### `[loadbalancer]` #### `ips` @@ -641,8 +629,6 @@ configured to consume; defaults to 1/8th of the total server memory. Comma-separated list of IP addresses or netmasks of external load balancers whose `X-Forwarded-For` should be respected. - - ### `[http_proxy]` #### `host` diff --git a/docs/production/email-gateway.md b/docs/production/email-gateway.md index 3dcb9cd670..391003fcde 100644 --- a/docs/production/email-gateway.md +++ b/docs/production/email-gateway.md @@ -18,10 +18,10 @@ that address will be delivered into the stream. There are two ways to configure Zulip's email gateway: - 1. Local delivery (recommended): A postfix server runs on the Zulip - server and passes the emails directly to Zulip. - 1. Polling: A cron job running on the Zulip server checks an IMAP - inbox (`username@example.com`) every minute for new emails. +1. Local delivery (recommended): A postfix server runs on the Zulip + server and passes the emails directly to Zulip. +1. Polling: A cron job running on the Zulip server checks an IMAP + inbox (`username@example.com`) every minute for new emails. The local delivery configuration is preferred for production because it supports nicer looking email addresses and has no cron delay. The @@ -57,31 +57,32 @@ using an [HTTP reverse proxy][reverse-proxy]). configuring email for `emaildomain.example.com` to be processed by `hostname.example.com`. You can check your work using this command: - ```console - $ dig +short emaildomain.example.com -t MX - 1 hostname.example.com - ``` + ```console + $ dig +short emaildomain.example.com -t MX + 1 hostname.example.com + ``` 1. Log in to your Zulip server; the remaining steps all happen there. 1. Add `, zulip::postfix_localmail` to `puppet_classes` in `/etc/zulip/zulip.conf`. A typical value after this change is: + ```ini puppet_classes = zulip::profile::standalone, zulip::postfix_localmail ``` -1. If `hostname.example.com` is different from +1. If `hostname.example.com` is different from `emaildomain.example.com`, add a section to `/etc/zulip/zulip.conf` on your Zulip server like this: - ```ini - [postfix] - mailname = emaildomain.example.com - ``` + ```ini + [postfix] + mailname = emaildomain.example.com + ``` - This tells postfix to expect to receive emails at addresses ending - with `@emaildomain.example.com`, overriding the default of - `@hostname.example.com`. + This tells postfix to expect to receive emails at addresses ending + with `@emaildomain.example.com`, overriding the default of + `@hostname.example.com`. 1. Run `/home/zulip/deployments/current/scripts/zulip-puppet-apply` (and answer `y`) to apply your new `/etc/zulip/zulip.conf` @@ -100,26 +101,27 @@ Congratulations! The integration should be fully operational. ## Polling setup 1. Create an email account dedicated to Zulip's email gateway - messages. We assume the address is of the form - `username@example.com`. The email provider needs to support the - standard model of delivering emails sent to - `username+stuff@example.com` to the `username@example.com` inbox. + messages. We assume the address is of the form + `username@example.com`. The email provider needs to support the + standard model of delivering emails sent to + `username+stuff@example.com` to the `username@example.com` inbox. 1. Edit `/etc/zulip/settings.py`, and set `EMAIL_GATEWAY_PATTERN` to `"username+%s@example.com"`. 1. Set up IMAP for your email account and obtain the authentication details. - ([Here's how it works with Gmail](https://support.google.com/mail/answer/7126229?hl=en)) + ([Here's how it works with Gmail](https://support.google.com/mail/answer/7126229?hl=en)) 1. Configure IMAP access in the appropriate Zulip settings: - - Login and server connection details in `/etc/zulip/settings.py` - in the email gateway integration section (`EMAIL_GATEWAY_LOGIN` and others). - - Password in `/etc/zulip/zulip-secrets.conf` as `email_gateway_password`. + + - Login and server connection details in `/etc/zulip/settings.py` + in the email gateway integration section (`EMAIL_GATEWAY_LOGIN` and others). + - Password in `/etc/zulip/zulip-secrets.conf` as `email_gateway_password`. 1. Install a cron job to poll the inbox every minute for new messages: - ```bash - cd /home/zulip/deployments/current/ - sudo cp puppet/zulip/files/cron.d/email-mirror /etc/cron.d/ - ``` + ```bash + cd /home/zulip/deployments/current/ + sudo cp puppet/zulip/files/cron.d/email-mirror /etc/cron.d/ + ``` Congratulations! The integration should be fully operational. diff --git a/docs/production/email.md b/docs/production/email.md index 26b08d03ec..03cb4dc310 100644 --- a/docs/production/email.md +++ b/docs/production/email.md @@ -100,6 +100,7 @@ We don't recommend using an inbox product like Gmail for outgoing email, because Gmail's anti-spam measures make this annoying. But if you want to use a Gmail account to send outgoing email anyway, here's how to make it work: + - Create a totally new Gmail account for your Zulip server; you don't want Zulip's automated emails to come from your personal email address. - If you're using 2-factor authentication on the Gmail account, you'll diff --git a/docs/production/expensive-migrations.md b/docs/production/expensive-migrations.md index 985c0b1352..398060ae0f 100644 --- a/docs/production/expensive-migrations.md +++ b/docs/production/expensive-migrations.md @@ -13,10 +13,10 @@ you'd like to watch the downtime phase of the upgrade closely, you can run them manually before starting the upgrade: 1. Log in to your Zulip server as the `zulip` user (or as `root` and - then run `su zulip` to drop privileges), and - `cd /home/zulip/deployments/current` + then run `su zulip` to drop privileges), and + `cd /home/zulip/deployments/current` 2. Run `./manage.py dbshell`. This will open a shell connected to the - PostgreSQL database. + PostgreSQL database. 3. In the PostgreSQL shell, run the following commands: ```postgresql diff --git a/docs/production/export-and-import.md b/docs/production/export-and-import.md index d78aae7ef6..53191a3c83 100644 --- a/docs/production/export-and-import.md +++ b/docs/production/export-and-import.md @@ -65,6 +65,7 @@ su zulip -c '/home/zulip/deployments/current/manage.py backup' ``` The backup tool provides the following options: + - `--output=/tmp/backup.tar.gz`: Filename to write the backup tarball to (default: write to a file in `/tmp`). On success, the console output will show the path to the output tarball. @@ -160,12 +161,12 @@ included in the backups generated by Zulip's standard tools. The data includes: - The PostgreSQL database. You can back this up with any standard -database export or backup tool. Zulip has built-in support for taking -daily incremental backups using -[wal-g](https://github.com/wal-g/wal-g); these backups are stored for -30 days in S3. If you have an Amazon S3 bucket you wish to store for -storing the backups, edit `/etc/zulip/zulip-secrets.conf` on the -PostgreSQL server to add: + database export or backup tool. Zulip has built-in support for taking + daily incremental backups using + [wal-g](https://github.com/wal-g/wal-g); these backups are stored for + 30 days in S3. If you have an Amazon S3 bucket you wish to store for + storing the backups, edit `/etc/zulip/zulip-secrets.conf` on the + PostgreSQL server to add: ```ini s3_backups_key = # aws public key @@ -180,17 +181,17 @@ PostgreSQL server to add: `/usr/lib/nagios/plugins/zulip_postgresql_backups/check_postgresql_backup`. - Any user-uploaded files. If you're using S3 as storage for file -uploads, this is backed up in S3. But if you have instead set -`LOCAL_UPLOADS_DIR`, any files uploaded by users (including avatars) -will be stored in that directory and you'll want to back it up. + uploads, this is backed up in S3. But if you have instead set + `LOCAL_UPLOADS_DIR`, any files uploaded by users (including avatars) + will be stored in that directory and you'll want to back it up. - Your Zulip configuration including secrets from `/etc/zulip/`. -E.g. if you lose the value of `secret_key`, all users will need to -log in again when you set up a replacement server since you won't be -able to verify their cookies. If you lose `avatar_salt`, any -user-uploaded avatars will need to be re-uploaded (since avatar -filenames are computed using a hash of `avatar_salt` and user's -email), etc. + E.g. if you lose the value of `secret_key`, all users will need to + log in again when you set up a replacement server since you won't be + able to verify their cookies. If you lose `avatar_salt`, any + user-uploaded avatars will need to be re-uploaded (since avatar + filenames are computed using a hash of `avatar_salt` and user's + email), etc. [export-import]: ../production/export-and-import.md @@ -258,17 +259,17 @@ before exporting; so that nobody can send new messages (etc.) while you're exporting data. There are two ways to do this: 1. `./scripts/stop-server`, which stops the whole server. This is -preferred if you're not hosting multiple organizations, because it has -no side effects other than disabling the Zulip server for the -duration. + preferred if you're not hosting multiple organizations, because it has + no side effects other than disabling the Zulip server for the + duration. 1. Pass `--deactivate` to `./manage export`, which first deactivates -the target organization, logging out all active login sessions and -preventing all accounts from logging in or accessing the API. This is -preferred for environments like Zulip Cloud where you might want to -export a single organization without disrupting any other users, and -the intent is to move hosting of the organization (and forcing users -to re-log in would be required as part of the hosting migration -anyway). + the target organization, logging out all active login sessions and + preventing all accounts from logging in or accessing the API. This is + preferred for environments like Zulip Cloud where you might want to + export a single organization without disrupting any other users, and + the intent is to move hosting of the organization (and forcing users + to re-log in would be required as part of the hosting migration + anyway). We include both options in the instructions below, commented out so that neither runs (using the `# ` at the start of the lines). If @@ -298,8 +299,9 @@ archive of all the organization's uploaded files. ## Import into a new Zulip server 1. [Install a new Zulip server](../production/install.md), -**skipping Step 3** (you'll create your Zulip organization via the data - import tool instead). + **skipping Step 3** (you'll create your Zulip organization via the data + import tool instead). + - Ensure that the Zulip server you're importing into is running the same version of Zulip as the server you're exporting from. @@ -321,10 +323,10 @@ archive of all the organization's uploaded files. budget extra RAM for running the data import tool. 2. If your new Zulip server is meant to fully replace a previous Zulip -server, you may want to copy some settings from `/etc/zulip` to your -new server to reuse the server-level configuration and secret keys -from your old server. There are a few important details to understand -about doing so: + server, you may want to copy some settings from `/etc/zulip` to your + new server to reuse the server-level configuration and secret keys + from your old server. There are a few important details to understand + about doing so: - Copying `/etc/zulip/settings.py` and `/etc/zulip/zulip.conf` is safe and recommended. Care is required when copying secrets from @@ -361,8 +363,8 @@ about doing so: avatar image improperly for users whose ID was renumbered. 3. Log in to a shell on your Zulip server as the `zulip` user. Run the -following commands, replacing the filename with the path to your data -export tarball: + following commands, replacing the filename with the path to your data + export tarball: ```bash cd ~ @@ -408,6 +410,7 @@ recommend starting with sending one to yourself for testing: ``` and then once you're ready, you can email them to everyone using e.g. + ```bash ./manage.py send_password_reset_email -r '' --all-users ``` @@ -434,6 +437,7 @@ realm.delete() The output contains details on the objects deleted from the database. Now, exit the management shell and run this to clear Zulip's cache: + ```bash /home/zulip/deployments/current/scripts/setup/flush-memcached ``` diff --git a/docs/production/giphy-gif-integration.md b/docs/production/giphy-gif-integration.md index c3e120ba03..1ae22259e2 100644 --- a/docs/production/giphy-gif-integration.md +++ b/docs/production/giphy-gif-integration.md @@ -41,7 +41,5 @@ Zulip server. Your users can now use the integration as described in [the help center article][help-center-giphy]. (A browser reload may be required). - - [help-center-giphy]: https://zulip.com/help/animated-gifs-from-giphy [giphy-dashboard]: https://developers.giphy.com/dashboard/ diff --git a/docs/production/install-existing-server.md b/docs/production/install-existing-server.md index cb064a9281..c0bd55ab0a 100644 --- a/docs/production/install-existing-server.md +++ b/docs/production/install-existing-server.md @@ -11,7 +11,7 @@ configuration files in /etc, so we recommend against installing it on a server running other nginx or django apps. But if you do, here are some things you can do that may make it -possible to retain your existing site. However, this is *NOT* +possible to retain your existing site. However, this is _NOT_ recommended, and you may break your server. Make sure you have backups and a provisioning script ready to go to wipe and restore your existing services if (when) your server goes down. diff --git a/docs/production/install.md b/docs/production/install.md index f5a9587714..910f381d6f 100644 --- a/docs/production/install.md +++ b/docs/production/install.md @@ -25,10 +25,10 @@ tar -xf zulip-server-latest.tar.gz ``` - If you'd like to verify the download, we -[publish the sha256sums of our release tarballs](https://www.zulip.org/dist/releases/SHA256SUMS.txt). + [publish the sha256sums of our release tarballs](https://www.zulip.org/dist/releases/SHA256SUMS.txt). - You can also -[install a pre-release version of Zulip](../production/deployment.html#installing-zulip-from-git) -using code from our [repository on GitHub](https://github.com/zulip/zulip/). + [install a pre-release version of Zulip](../production/deployment.html#installing-zulip-from-git) + using code from our [repository on GitHub](https://github.com/zulip/zulip/). ## Step 2: Install Zulip @@ -100,6 +100,7 @@ do. To really see Zulip in action, you'll need to get the people you work together with using it with you. + - [Set up outgoing email](email.md) so Zulip can confirm new users' email addresses and send notifications. - Learn how to [get your organization started][realm-admin-docs] using @@ -108,36 +109,38 @@ together with using it with you. Learning more: - Subscribe to the [Zulip announcements email -list](https://groups.google.com/forum/#!forum/zulip-announce) for -server administrators. This extremely low-traffic list is for -important announcements, including [new -releases](../overview/release-lifecycle.md) and security issues. You -can also use the [RSS -feed](https://groups.google.com/forum/#!aboutgroup/zulip-announce). + list](https://groups.google.com/forum/#!forum/zulip-announce) for + server administrators. This extremely low-traffic list is for + important announcements, including [new + releases](../overview/release-lifecycle.md) and security issues. You + can also use the [RSS + feed](https://groups.google.com/forum/#!aboutgroup/zulip-announce). - Follow [Zulip on Twitter](https://twitter.com/zulip). - Learn how to [configure your Zulip server settings](settings.md). - Learn about [Backups, export and import](../production/export-and-import.md) -and [upgrading](../production/upgrade-or-modify.md) a production Zulip -server. + and [upgrading](../production/upgrade-or-modify.md) a production Zulip + server. [realm-admin-docs]: https://zulip.com/help/getting-your-organization-started-with-zulip ```eval_rst .. _installer-details: ``` + ## Details: What the installer does The install script does several things: + - Creates the `zulip` user, which the various Zulip servers will run as. - Creates `/home/zulip/deployments/`, which the Zulip code for this -deployment (and future deployments when you upgrade) goes into. At the -very end of the install process, the script moves the Zulip code tree -it's running from (which you unpacked from a tarball above) to a -directory there, and makes `/home/zulip/deployments/current` as a -symbolic link to it. + deployment (and future deployments when you upgrade) goes into. At the + very end of the install process, the script moves the Zulip code tree + it's running from (which you unpacked from a tarball above) to a + directory there, and makes `/home/zulip/deployments/current` as a + symbolic link to it. - Installs Zulip's various dependencies. - Configures the various third-party services Zulip uses, including -PostgreSQL, RabbitMQ, Memcached and Redis. + PostgreSQL, RabbitMQ, Memcached and Redis. - Initializes Zulip's database. If you'd like to deploy Zulip with these services on different diff --git a/docs/production/maintain-secure-upgrade.md b/docs/production/maintain-secure-upgrade.md index b0b3a65a4d..4f0e311e5c 100644 --- a/docs/production/maintain-secure-upgrade.md +++ b/docs/production/maintain-secure-upgrade.md @@ -8,6 +8,7 @@ This was once a long page covering a bunch of topics; those topics have since all moved to dedicated pages: ### Monitoring + Moved to [Troubleshooting](../production/troubleshooting.html#monitoring). ### Securing your Zulip server diff --git a/docs/production/mobile-push-notifications.md b/docs/production/mobile-push-notifications.md index 58326e2412..b204638af1 100644 --- a/docs/production/mobile-push-notifications.md +++ b/docs/production/mobile-push-notifications.md @@ -11,8 +11,6 @@ will need to register your Zulip server with the Zulip mobile push notification service. This service will forward push notifications generated by your server to the Zulip mobile app automatically. - - ## How to sign up Starting with Zulip 1.6 for both Android and iOS, Zulip servers @@ -34,16 +32,18 @@ You can enable this for your Zulip server as follows: need to add the line (it won't be there to uncomment). 1. If you're running Zulip 1.8.1 or newer, you can run the - registration command: - ```bash - # As root: - su zulip -c '/home/zulip/deployments/current/manage.py register_server' - # Or as the zulip user, you can skip the `su zulip -c`: - /home/zulip/deployments/current/manage.py register_server + registration command: + + ```bash + # As root: + su zulip -c '/home/zulip/deployments/current/manage.py register_server' + # Or as the zulip user, you can skip the `su zulip -c`: + /home/zulip/deployments/current/manage.py register_server + + # docker-zulip users can run this inside the container with `docker exec`: + docker exec -it -u zulip /home/zulip/deployments/current/manage.py register_server + ``` - # docker-zulip users can run this inside the container with `docker exec`: - docker exec -it -u zulip /home/zulip/deployments/current/manage.py register_server - ``` This command will print the registration data it would send to the mobile push notifications service, ask you to accept the terms of service, and if you accept, register your server. If you have trouble, @@ -62,15 +62,15 @@ following. Please follow the instructions carefully: (normally they're only sent if you're idle, which isn't ideal for this sort of testing). - On an Android device, download and log in to the -[Zulip Android app](https://play.google.com/store/apps/details?id=com.zulipmobile). -If you were already logged in before configuring the server, you'll -need to log out first, since the app only registers for push -notifications on login. + [Zulip Android app](https://play.google.com/store/apps/details?id=com.zulipmobile). + If you were already logged in before configuring the server, you'll + need to log out first, since the app only registers for push + notifications on login. - Hit the home button, so Zulip is running in the background, and then -have **another user** send you a **private message** (By default, -Zulip only sends push notifications for private messages sent by other -users and messages mentioning you). A push notification should appear -in the Android notification area. + have **another user** send you a **private message** (By default, + Zulip only sends push notifications for private messages sent by other + users and messages mentioning you). A push notification should appear + in the Android notification area. [mobile-notifications-always]: https://zulip.com/help/test-mobile-notifications @@ -119,26 +119,27 @@ and privacy in mind: - The Push Notification Service only stores the necessary metadata for delivering the notifications to the appropriate devices, and nothing else: - - The APNS/FCM tokens needed to securely send mobile push - notifications to iOS and Android devices, one per device - registered to be notified by your Zulip server. - - User ID numbers generated by your Zulip server, needed to route - a given notification to the appropriate set of mobile devices. - These user ID numbers are are opaque to the Push Notification - Service and Kandra Labs. + - The APNS/FCM tokens needed to securely send mobile push + notifications to iOS and Android devices, one per device + registered to be notified by your Zulip server. + - User ID numbers generated by your Zulip server, needed to route + a given notification to the appropriate set of mobile devices. + These user ID numbers are are opaque to the Push Notification + Service and Kandra Labs. - The Push Notification Service receives (but does not store) the contents of individual mobile push notifications: - - The numeric message ID generated by your Zulip server. - - Metadata on the message's sender (name and avatar URL). - - Metadata on the message's recipient (stream name + ID, topic, - private message recipients, etc.). - - A timestamp. - - The message's content. + + - The numeric message ID generated by your Zulip server. + - Metadata on the message's sender (name and avatar URL). + - Metadata on the message's recipient (stream name + ID, topic, + private message recipients, etc.). + - A timestamp. + - The message's content. There's a `PUSH_NOTIFICATION_REDACT_CONTENT` setting available to disable any message content being sent via the push notification bouncer (i.e. message content will be replaced with - `***REDACTED***`). Note that this setting makes push notifications + `***REDACTED***`). Note that this setting makes push notifications significantly less usable. We plan to @@ -146,6 +147,7 @@ and privacy in mind: which would eliminate that usability tradeoff and additionally allow us to not have any access to the other details mentioned in this section. + - All of the network requests (both from Zulip servers to the Push Notification Service and from the Push Notification Service to the relevant Google and Apple services) are encrypted over the wire with @@ -217,6 +219,7 @@ the app stores yourself. If you've done that work, the Zulip server configuration for sending push notifications through the new app is quite straightforward: + - Create a [FCM push notifications](https://firebase.google.com/docs/cloud-messaging) key in the Google Developer console and set `android_gcm_api_key` in diff --git a/docs/production/password-strength.md b/docs/production/password-strength.md index 431b57301c..8b977d3458 100644 --- a/docs/production/password-strength.md +++ b/docs/production/password-strength.md @@ -14,7 +14,7 @@ detail how we set the default threshold (`PASSWORD_MIN_GUESSES`) we use. First, read the doc section there. (It's short.) Then, the CACM article ["Passwords and the Evolution of Imperfect -Authentication"][BHOS15] is comprehensive, educational, and readable, +Authentication"][bhos15] is comprehensive, educational, and readable, and is especially recommended. The CACM article is convincing that password requirements should be @@ -36,7 +36,7 @@ overestimation (allowing a weak password) sharply degrades at 100k guesses, while underestimation (rejecting a strong password) jumps up just after 10k guesses, and grows steadily thereafter. -Moreover, the [Yahoo study][Bon12] shows that resistance to even 1M +Moreover, the [Yahoo study][bon12] shows that resistance to even 1M guesses is more than nearly half of users accomplish with a freely chosen password, and 100k is too much for about 20%. (See Figure 6.) It doesn't make sense for a Zulip server to try to educate or push so @@ -54,6 +54,6 @@ zxcvbn rarely underestimates the strength of a password too severely, and only about 10% of users do worse than this without prompting. [zxcvbn]: https://github.com/dropbox/zxcvbn -[BHOS15]: https://www.cl.cam.ac.uk/~fms27/papers/2015-BonneauHerOorSta-passwords.pdf +[bhos15]: https://www.cl.cam.ac.uk/~fms27/papers/2015-BonneauHerOorSta-passwords.pdf [zxcvbn-paper]: https://www.usenix.org/system/files/conference/usenixsecurity16/sec16_paper_wheeler.pdf -[Bon12]: https://ieeexplore.ieee.org/document/6234435 +[bon12]: https://ieeexplore.ieee.org/document/6234435 diff --git a/docs/production/postgresql.md b/docs/production/postgresql.md index 7e4d984d62..42d10e9763 100644 --- a/docs/production/postgresql.md +++ b/docs/production/postgresql.md @@ -1,5 +1,4 @@ -PostgreSQL database details -========================= +# PostgreSQL database details Starting with Zulip 3.0, Zulip supports a range of PostgreSQL versions. PostgreSQL 13 is the current default for new installations; @@ -140,7 +139,6 @@ stop PostgreSQL and restart it using pg_ctlcluster after you've debugged with this approach, since it does bypass some of the work that pg_ctlcluster does. - #### PostgreSQL vacuuming alerts The `autovac_freeze` PostgreSQL alert from `check_postgres` is diff --git a/docs/production/requirements.md b/docs/production/requirements.md index 846669af7f..088e22e54b 100644 --- a/docs/production/requirements.md +++ b/docs/production/requirements.md @@ -1,6 +1,7 @@ # Requirements and scalability To run a Zulip server, you will need: + - A dedicated machine or VM - A supported OS: - Ubuntu 20.04 Focal @@ -44,8 +45,7 @@ sudo apt update ``` [upgrade-os]: ../production/upgrade-or-modify.html#upgrading-the-operating-system -[ubuntu-repositories]: -https://help.ubuntu.com/community/Repositories/Ubuntu +[ubuntu-repositories]: https://help.ubuntu.com/community/Repositories/Ubuntu [enable-universe]: https://help.ubuntu.com/community/Repositories/CommandLine#Adding_the_Universe_and_Multiverse_Repositories #### Hardware specifications @@ -93,10 +93,10 @@ on hardware requirements for larger organizations. - Zulip supports [running behind a reverse proxy][reverse-proxy]. - Zulip servers running inside a private network should configure the [Smokescreen integration][smokescreen-proxy] to protect against - [SSRF attacks][SSRF], where users could make the Zulip server make + [SSRF attacks][ssrf], where users could make the Zulip server make requests to private resources. -[SSRF]: https://owasp.org/www-community/attacks/Server_Side_Request_Forgery +[ssrf]: https://owasp.org/www-community/attacks/Server_Side_Request_Forgery [smokescreen-proxy]: ../production/deployment.html#using-an-outgoing-http-proxy [reverse-proxy]: ../production/deployment.html#putting-the-zulip-application-behind-a-reverse-proxy [email-mirror-code]: https://github.com/zulip/zulip/blob/main/zerver/management/commands/email_mirror.py @@ -140,8 +140,9 @@ Zulip in production](../production/install.md). This section details some basic guidelines for running a Zulip server for larger organizations (especially >1000 users or 500+ daily active users). Zulip's resource needs depend mainly on 3 parameters: + - daily active users (e.g. number of employees if everyone's an -employee) + employee) - total user accounts (can be much larger) - message volume. @@ -159,16 +160,17 @@ installing Zulip with a dedicated database server. active users, we recommend using a [remote PostgreSQL database](postgresql.md), but it's not required. -- **RAM:** We recommended more RAM for larger installations: - - With 25+ daily active users, 4GB of RAM. - - With 100+ daily active users, 8GB of RAM. - - With 400+ daily active users, 16GB of RAM for the Zulip - application server, plus 16GB for the database. - - With 2000+ daily active users 32GB of RAM, plus 32GB for the - database. - - Roughly linear scaling beyond that. +- **RAM:** We recommended more RAM for larger installations: -- **CPU:** The Zulip application server's CPU usage is heavily + - With 25+ daily active users, 4GB of RAM. + - With 100+ daily active users, 8GB of RAM. + - With 400+ daily active users, 16GB of RAM for the Zulip + application server, plus 16GB for the database. + - With 2000+ daily active users 32GB of RAM, plus 32GB for the + database. + - Roughly linear scaling beyond that. + +- **CPU:** The Zulip application server's CPU usage is heavily optimized due to extensive work on optimizing the performance of requests for latency reasons. Because most servers with sufficient RAM have sufficient CPU resources, CPU requirements are rarely an diff --git a/docs/production/security-model.md b/docs/production/security-model.md index 152d043ab0..6caf998374 100644 --- a/docs/production/security-model.md +++ b/docs/production/security-model.md @@ -70,10 +70,10 @@ strength allowed is controlled by two settings in Estimating the guessability of a password is a complex problem and impossible to efficiently do perfectly. For background or when considering an alternate value for this setting, the article - ["Passwords and the Evolution of Imperfect Authentication"][BHOS15] + ["Passwords and the Evolution of Imperfect Authentication"][bhos15] is recommended. The [2016 zxcvbn paper][zxcvbn-paper] adds useful information about the performance of zxcvbn, and [a large 2012 study - of Yahoo users][Bon12] is informative about the strength of the + of Yahoo users][bon12] is informative about the strength of the passwords users choose. [zxcvbn]: https://github.com/dropbox/zxcvbn -[BHOS15]: http://www.cl.cam.ac.uk/~fms27/papers/2015-BonneauHerOorSta-passwords.pdf +[bhos15]: http://www.cl.cam.ac.uk/~fms27/papers/2015-BonneauHerOorSta-passwords.pdf [zxcvbn-paper]: https://www.usenix.org/system/files/conference/usenixsecurity16/sec16_paper_wheeler.pdf -[Bon12]: http://ieeexplore.ieee.org/document/6234435/ +[bon12]: http://ieeexplore.ieee.org/document/6234435/ ## Messages and history @@ -97,6 +97,7 @@ strength allowed is controlled by two settings in attacks. - Zulip supports both public streams and private streams. + - Any non-guest user can join any public stream in the organization, and can view the complete message history of any public stream without joining the stream. Guests can only access streams that @@ -258,11 +259,11 @@ strength allowed is controlled by two settings in the Zulip server to make HTTP requests on their behalf. As a result, Zulip supports routing all outgoing outgoing HTTP requests [through Smokescreen][smokescreen-setup] to ensure that Zulip cannot be - used to execute [SSRF attacks][SSRF] against other systems on an + used to execute [SSRF attacks][ssrf] against other systems on an internal corporate network. The default Smokescreen configuration denies access to all non-public IP addresses, including 127.0.0.1. -[SSRF]: https://owasp.org/www-community/attacks/Server_Side_Request_Forgery +[ssrf]: https://owasp.org/www-community/attacks/Server_Side_Request_Forgery [smokescreen-setup]: ../production/deployment.html#using-an-outgoing-http-proxy ## Final notes and security response diff --git a/docs/production/settings.md b/docs/production/settings.md index 79db332e8c..79991b46c2 100644 --- a/docs/production/settings.md +++ b/docs/production/settings.md @@ -15,6 +15,7 @@ This page discusses additional configuration that a system administrator can do. To change any of the following settings, edit the `/etc/zulip/settings.py` file on your Zulip server, and then restart the server with the following command: + ```bash su zulip -c '/home/zulip/deployments/current/scripts/restart-server' ``` @@ -95,6 +96,7 @@ configuration along with your other Zulip server configuration. ### Miscellaneous server settings Some popular settings in `/etc/zulip/settings.py` include: + - The Twitter integration, which provides pretty inline previews of tweets. - The [email gateway](../production/email-gateway.md), which lets diff --git a/docs/production/ssl-certificates.md b/docs/production/ssl-certificates.md index 347cb8f715..44b5993c18 100644 --- a/docs/production/ssl-certificates.md +++ b/docs/production/ssl-certificates.md @@ -11,6 +11,7 @@ chore (nor expense) that it used to be. If you already have an SSL certificate, just install (or symlink) its files into place at the following paths: + - `/etc/ssl/private/zulip.key` for the private key - `/etc/ssl/certs/zulip.combined-chain.crt` for the certificate. @@ -55,6 +56,7 @@ SSL certificates from Let's Encrypt and renew them automatically. We recommend most Zulip servers use Certbot. You'll want something else if: + - you have an existing workflow for managing SSL certificates that you prefer; - you need wildcard certificates (support from Let's Encrypt released @@ -85,10 +87,12 @@ one as described in the section below after installing Zulip. To enable the Certbot automation on an already-installed Zulip server, run the following commands: + ```bash sudo -s # If not already root /home/zulip/deployments/current/scripts/setup/setup-certbot --email=EMAIL HOSTNAME [HOSTNAME2...] ``` + where HOSTNAME is the domain name users see in their browser when using the server (e.g., `zulip.example.com`), and EMAIL is a contact address for the server admins. Additional hostnames can also be @@ -109,7 +113,6 @@ checks if any certificates are due for renewal, and if they are (so approximately once every 60 days), repeats the process of request, prove, get a fresh certificate. - ## Self-signed certificate If you aren't able to use Certbot, you can generate a self-signed SSL @@ -126,22 +129,24 @@ just pass the `--self-signed-cert` flag when To generate a self-signed certificate for an already-installed Zulip server, run the following commands: + ```bash sudo -s # If not already root /home/zulip/deployments/current/scripts/setup/generate-self-signed-cert HOSTNAME ``` + where HOSTNAME is the domain name (or IP address) to use on the generated certificate. After replacing the certificates, you need to reload `nginx` by running the following as `root`: + ```bash service nginx reload ``` [desktop-certs]: https://zulip.com/help/custom-certificates - ## Troubleshooting ### The Android app can't connect to the server @@ -149,7 +154,6 @@ service nginx reload This is most often caused by an incomplete certificate chain. See discussion in the [Manual install](#manual-install) section above. - ### The iOS app can't connect to the server This can be caused by a server set up to support only TLS 1.1 or @@ -157,7 +161,7 @@ older (including TLS 1.0, SSL 3, or SSL 2.) TLS 1.2 has been a standard for over 10 years, and all modern web server software supports it. Starting in early 2020, all major -browsers [will *require* TLS 1.2 or later][tls12-required-news], and +browsers [will _require_ TLS 1.2 or later][tls12-required-news], and will refuse to connect over TLS 1.1 or older. And on iOS, Apple [has since iOS 9][apple-ats] required TLS 1.2 for all connections made by apps, unless the app specifically opts into lower security. @@ -175,7 +179,6 @@ directive][nginx-doc-protocols] in your configuration. [nginx-doc-protocols]: https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_protocols - ### The Android app connects to the server on some devices but not others An issue on Android 7.0 ([report][android7.0-tls-issue], diff --git a/docs/production/troubleshooting.md b/docs/production/troubleshooting.md index 6bb19ba027..47081125c0 100644 --- a/docs/production/troubleshooting.md +++ b/docs/production/troubleshooting.md @@ -37,6 +37,7 @@ and restart various services. ### Checking status with `supervisorctl status` You can check if the Zulip application is running using: + ```bash supervisorctl status ``` @@ -111,15 +112,18 @@ problems and how to resolve them: nginx will fail to start if you configured SSL incorrectly or did not provide SSL certificates. To fix this, configure them properly and then run: + ```bash service nginx restart ``` - If your host is being port scanned by unauthorized users, you may see messages in `/var/log/zulip/server.log` like + ```text 2017-02-22 14:11:33,537 ERROR Invalid HTTP_HOST header: '10.2.3.4'. You may need to add u'10.2.3.4' to ALLOWED_HOSTS. ``` + Django uses the hostnames configured in `ALLOWED_HOSTS` to identify legitimate requests and block others. When an incoming request does not have the correct HTTP Host header, Django rejects it and logs the diff --git a/docs/production/upgrade-or-modify.md b/docs/production/upgrade-or-modify.md index 8599b178da..2a1bc3ca89 100644 --- a/docs/production/upgrade-or-modify.md +++ b/docs/production/upgrade-or-modify.md @@ -24,33 +24,34 @@ to a new Zulip release: for all releases newer than what is currently installed. 1. Download the appropriate release tarball from - You can download the latest - release with: + You can download the latest + release with: - ```bash - wget https://www.zulip.org/dist/releases/zulip-server-latest.tar.gz - ``` + ```bash + wget https://www.zulip.org/dist/releases/zulip-server-latest.tar.gz + ``` - You also have the option of upgrading Zulip [to a version in a Git - repository directly](#upgrading-from-a-git-repository) or creating - your own release tarballs from a copy of the [zulip.git - repository](https://github.com/zulip/zulip) using - `tools/build-release-tarball`. + You also have the option of upgrading Zulip [to a version in a Git + repository directly](#upgrading-from-a-git-repository) or creating + your own release tarballs from a copy of the [zulip.git + repository](https://github.com/zulip/zulip) using + `tools/build-release-tarball`. 1. Log in to your Zulip and run as root: - ```bash - /home/zulip/deployments/current/scripts/upgrade-zulip zulip-server-VERSION.tar.gz - ``` + ```bash + /home/zulip/deployments/current/scripts/upgrade-zulip zulip-server-VERSION.tar.gz + ``` - The upgrade process will: - - Run `apt-get upgrade` - - Install new versions of Zulip's dependencies (mainly Python packages). - - (`upgrade-zulip-from-git` only) Build Zulip's frontend assets using `webpack`. - - Shut down the Zulip service - - Run a `puppet apply` - - Run any database migrations - - Bring the Zulip service back up on the new version. + The upgrade process will: + + - Run `apt-get upgrade` + - Install new versions of Zulip's dependencies (mainly Python packages). + - (`upgrade-zulip-from-git` only) Build Zulip's frontend assets using `webpack`. + - Shut down the Zulip service + - Run a `puppet apply` + - Run any database migrations + - Bring the Zulip service back up on the new version. Upgrading will result in brief downtime for the service, which should be under 30 seconds unless there is an expensive database migration @@ -176,6 +177,7 @@ after resolving an issue. The most common causes of errors are: the upgrade process. Useful logs are available in a few places: + - The Zulip upgrade scripts log all output to `/var/log/zulip/upgrade.log`. - The Zulip server logs all Internal Server Errors to @@ -261,36 +263,36 @@ instructions for other supported platforms. 2. As the Zulip user, stop the Zulip server and run the following to back up the system: - ```bash - supervisorctl stop all - /home/zulip/deployments/current/manage.py backup --output=/home/zulip/release-upgrade.backup.tar.gz - ``` + ```bash + supervisorctl stop all + /home/zulip/deployments/current/manage.py backup --output=/home/zulip/release-upgrade.backup.tar.gz + ``` 3. Switch to the root user and upgrade the operating system using the OS's standard tooling. E.g. for Ubuntu, this means running `do-release-upgrade` and following the prompts until it completes successfully: - ```bash - sudo -i # Or otherwise get a root shell - do-release-upgrade -d - ``` + ```bash + sudo -i # Or otherwise get a root shell + do-release-upgrade -d + ``` - The `-d` option to `do-release-upgrade` is required because Ubuntu - 20.04 is new; it will stop being necessary once the first point - release update of Ubuntu 20.04 LTS is released. + The `-d` option to `do-release-upgrade` is required because Ubuntu + 20.04 is new; it will stop being necessary once the first point + release update of Ubuntu 20.04 LTS is released. - When `do-release-upgrade` asks you how to upgrade configuration - files for services that Zulip manages like Redis, PostgreSQL, - Nginx, and memcached, the best choice is `N` to keep the - currently installed version. But it's not important; the next - step will re-install Zulip's configuration in any case. + When `do-release-upgrade` asks you how to upgrade configuration + files for services that Zulip manages like Redis, PostgreSQL, + Nginx, and memcached, the best choice is `N` to keep the + currently installed version. But it's not important; the next + step will re-install Zulip's configuration in any case. 4. As root, upgrade the database to the latest version of PostgreSQL: - ```bash - /home/zulip/deployments/current/scripts/setup/upgrade-postgresql - ``` + ```bash + /home/zulip/deployments/current/scripts/setup/upgrade-postgresql + ``` 5. Ubuntu 20.04 has a different version of the low-level glibc library, which affects how PostgreSQL orders text data (known as @@ -307,11 +309,11 @@ instructions for other supported platforms. full-text search indexes to work with the upgraded dictionary packages: - ```bash - rm -rf /srv/zulip-venv-cache/* - /home/zulip/deployments/current/scripts/lib/upgrade-zulip-stage-2 \ - /home/zulip/deployments/current/ --ignore-static-assets --audit-fts-indexes - ``` + ```bash + rm -rf /srv/zulip-venv-cache/* + /home/zulip/deployments/current/scripts/lib/upgrade-zulip-stage-2 \ + /home/zulip/deployments/current/ --ignore-static-assets --audit-fts-indexes + ``` This will finish by restarting your Zulip server; you should now be able to navigate to its URL and confirm everything is working @@ -330,27 +332,27 @@ instructions for other supported platforms. 4. As root, upgrade the database installation and OS configuration to match the new OS version: - ```bash - touch /usr/share/postgresql/10/pgroonga_setup.sql.applied - /home/zulip/deployments/current/scripts/zulip-puppet-apply -f - pg_dropcluster 10 main --stop - systemctl stop postgresql - pg_upgradecluster 9.5 main - pg_dropcluster 9.5 main - apt remove postgresql-9.5 - systemctl start postgresql - systemctl restart memcached - ``` + ```bash + touch /usr/share/postgresql/10/pgroonga_setup.sql.applied + /home/zulip/deployments/current/scripts/zulip-puppet-apply -f + pg_dropcluster 10 main --stop + systemctl stop postgresql + pg_upgradecluster 9.5 main + pg_dropcluster 9.5 main + apt remove postgresql-9.5 + systemctl start postgresql + systemctl restart memcached + ``` 5. Finally, we need to reinstall the current version of Zulip, which among other things will recompile Zulip's Python module dependencies for your new version of Python: - ```bash - rm -rf /srv/zulip-venv-cache/* - /home/zulip/deployments/current/scripts/lib/upgrade-zulip-stage-2 \ - /home/zulip/deployments/current/ --ignore-static-assets - ``` + ```bash + rm -rf /srv/zulip-venv-cache/* + /home/zulip/deployments/current/scripts/lib/upgrade-zulip-stage-2 \ + /home/zulip/deployments/current/ --ignore-static-assets + ``` This will finish by restarting your Zulip server; you should now be able to navigate to its URL and confirm everything is working @@ -361,9 +363,9 @@ instructions for other supported platforms. 7. As root, finish by verifying the contents of the full-text indexes: - ```bash - /home/zulip/deployments/current/manage.py audit_fts_indexes - ``` + ```bash + /home/zulip/deployments/current/manage.py audit_fts_indexes + ``` ### Upgrading from Ubuntu 14.04 Trusty to 16.04 Xenial @@ -378,27 +380,27 @@ instructions for other supported platforms. 4. As root, upgrade the database installation and OS configuration to match the new OS version: - ```bash - apt remove upstart -y - /home/zulip/deployments/current/scripts/zulip-puppet-apply -f - pg_dropcluster 9.5 main --stop - systemctl stop postgresql - pg_upgradecluster -m upgrade 9.3 main - pg_dropcluster 9.3 main - apt remove postgresql-9.3 - systemctl start postgresql - service memcached restart - ``` + ```bash + apt remove upstart -y + /home/zulip/deployments/current/scripts/zulip-puppet-apply -f + pg_dropcluster 9.5 main --stop + systemctl stop postgresql + pg_upgradecluster -m upgrade 9.3 main + pg_dropcluster 9.3 main + apt remove postgresql-9.3 + systemctl start postgresql + service memcached restart + ``` 5. Finally, we need to reinstall the current version of Zulip, which among other things will recompile Zulip's Python module dependencies for your new version of Python: - ```bash - rm -rf /srv/zulip-venv-cache/* - /home/zulip/deployments/current/scripts/lib/upgrade-zulip-stage-2 \ - /home/zulip/deployments/current/ --ignore-static-assets - ``` + ```bash + rm -rf /srv/zulip-venv-cache/* + /home/zulip/deployments/current/scripts/lib/upgrade-zulip-stage-2 \ + /home/zulip/deployments/current/ --ignore-static-assets + ``` This will finish by restarting your Zulip server; you should now be able to navigate to its URL and confirm everything is working @@ -429,27 +431,27 @@ instructions for other supported platforms. 4. As root, upgrade the database installation and OS configuration to match the new OS version: - ```bash - apt remove upstart -y - /home/zulip/deployments/current/scripts/zulip-puppet-apply -f - pg_dropcluster 11 main --stop - systemctl stop postgresql - pg_upgradecluster -m upgrade 9.6 main - pg_dropcluster 9.6 main - apt remove postgresql-9.6 - systemctl start postgresql - service memcached restart - ``` + ```bash + apt remove upstart -y + /home/zulip/deployments/current/scripts/zulip-puppet-apply -f + pg_dropcluster 11 main --stop + systemctl stop postgresql + pg_upgradecluster -m upgrade 9.6 main + pg_dropcluster 9.6 main + apt remove postgresql-9.6 + systemctl start postgresql + service memcached restart + ``` 5. Finally, we need to reinstall the current version of Zulip, which among other things will recompile Zulip's Python module dependencies for your new version of Python: - ```bash - rm -rf /srv/zulip-venv-cache/* - /home/zulip/deployments/current/scripts/lib/upgrade-zulip-stage-2 \ - /home/zulip/deployments/current/ --ignore-static-assets - ``` + ```bash + rm -rf /srv/zulip-venv-cache/* + /home/zulip/deployments/current/scripts/lib/upgrade-zulip-stage-2 \ + /home/zulip/deployments/current/ --ignore-static-assets + ``` This will finish by restarting your Zulip server; you should now be able to navigate to its URL and confirm everything is working @@ -469,9 +471,9 @@ instructions for other supported platforms. 8. As root, finish by verifying the contents of the full-text indexes: - ```bash - /home/zulip/deployments/current/manage.py audit_fts_indexes - ``` + ```bash + /home/zulip/deployments/current/manage.py audit_fts_indexes + ``` ## Upgrading PostgreSQL @@ -487,34 +489,33 @@ To upgrade the version of PostgreSQL on the Zulip server: 1. Stop the server, as the `zulip` user: - ```bash - # On Zulip before 4.0, use `supervisor stop all` instead - /home/zulip/deployments/current/scripts/stop-server - ``` + ```bash + # On Zulip before 4.0, use `supervisor stop all` instead + /home/zulip/deployments/current/scripts/stop-server + ``` 1. Take a backup, in case of any problems: - ```bash - /home/zulip/deployments/current/manage.py backup --output=/home/zulip/postgresql-upgrade.backup.tar.gz - ``` + ```bash + /home/zulip/deployments/current/manage.py backup --output=/home/zulip/postgresql-upgrade.backup.tar.gz + ``` 1. As root, run the database upgrade tool: - ```bash - /home/zulip/deployments/current/scripts/setup/upgrade-postgresql - ``` + ```bash + /home/zulip/deployments/current/scripts/setup/upgrade-postgresql + ``` 1. As the `zulip` user, start the server again: - ```bash - # On Zulip before 4.0, use `restart-server` instead of `start-server` instead - /home/zulip/deployments/current/scripts/start-server - ``` + ```bash + # On Zulip before 4.0, use `restart-server` instead of `start-server` instead + /home/zulip/deployments/current/scripts/start-server + ``` You should now be able to navigate to the Zulip server's URL and confirm everything is working correctly. - ## Modifying Zulip Zulip is 100% free and open source software, and you're welcome to @@ -526,10 +527,10 @@ modified version of Zulip, please be responsible about communicating that fact: - Ideally, you'd reproduce the issue in an unmodified version (e.g. on -[chat.zulip.org](../contributing/chat-zulip-org.md) or -[zulip.com](https://zulip.com)). + [chat.zulip.org](../contributing/chat-zulip-org.md) or + [zulip.com](https://zulip.com)). - Where that is difficult or you think it's very unlikely your changes -are related to the issue, just mention your changes in the issue report. + are related to the issue, just mention your changes in the issue report. If you're looking to modify Zulip by applying changes developed by the Zulip core team and merged into `main`, skip to [this @@ -589,9 +590,9 @@ git push origin +acme-branch ``` - Log in to your Zulip server and configure and use -[upgrade-zulip-from-git][] to install the changes; remember to -configure `git_repo_url` to point to your fork on GitHub and run it as -`upgrade-zulip-from-git acme-branch`. + [upgrade-zulip-from-git][] to install the changes; remember to + configure `git_repo_url` to point to your fork on GitHub and run it as + `upgrade-zulip-from-git acme-branch`. This workflow solves all of the problems described above: your change will be compiled and installed correctly (restarting the server), and diff --git a/docs/production/upload-backends.md b/docs/production/upload-backends.md index 65f15cac59..6c9abb8597 100644 --- a/docs/production/upload-backends.md +++ b/docs/production/upload-backends.md @@ -21,10 +21,10 @@ Here, we document the process for configuring Zulip's S3 file upload backend. To enable this backend, you need to do the following: 1. In the AWS management console, create a new IAM account (aka API -user) for your Zulip server, and two buckets in S3, one for uploaded -files included in messages, and another for user avatars. You need -two buckets because the "user avatars" bucket is generally configured -as world-readable, whereas the "uploaded files" one is not. + user) for your Zulip server, and two buckets in S3, one for uploaded + files included in messages, and another for user avatars. You need + two buckets because the "user avatars" bucket is generally configured + as world-readable, whereas the "uploaded files" one is not. 1. Set `s3_key` and `s3_secret_key` in /etc/zulip/zulip-secrets.conf to be the S3 access and secret keys for the IAM account. @@ -140,18 +140,18 @@ your local backend to Amazon S3. Follow these instructions, step by step, to do the migration. 1. First, [set up the S3 backend](#s3-backend-configuration) in the settings - (all the auth stuff), but leave `LOCAL_UPLOADS_DIR` set -- the - migration tool will need that value to know where to find your uploads. + (all the auth stuff), but leave `LOCAL_UPLOADS_DIR` set -- the + migration tool will need that value to know where to find your uploads. 2. Run `./manage.py transfer_uploads_to_s3`. This will upload all the - files from the local uploads directory to Amazon S3. By default, - this command runs on 6 parallel processes, since uploading is a - latency-sensitive operation. You can control this parameter using - the `--processes` option. + files from the local uploads directory to Amazon S3. By default, + this command runs on 6 parallel processes, since uploading is a + latency-sensitive operation. You can control this parameter using + the `--processes` option. 3. Once the transfer script completes, disable `LOCAL_UPLOADS_DIR`, and - restart your server (continuing the last few steps of the S3 - backend setup instructions). + restart your server (continuing the last few steps of the S3 + backend setup instructions). Congratulations! Your uploaded files are now migrated to S3. **Caveat**: The current version of this tool does not migrate an - uploaded organization avatar or logo. +uploaded organization avatar or logo. diff --git a/docs/subsystems/analytics.md b/docs/subsystems/analytics.md index 067d04e5a8..fa38bbfae5 100644 --- a/docs/subsystems/analytics.md +++ b/docs/subsystems/analytics.md @@ -29,7 +29,7 @@ There are three main components: The next several sections will dive into the details of these components. -## The *Count database tables +## The \*Count database tables The Zulip analytics system is built around collecting time series data in a set of database tables. Each of these tables has the following fields: @@ -76,7 +76,7 @@ by the system and with what data. ## The FillState table The default Zulip production configuration runs a cron job once an hour that -updates the *Count tables for each of the CountStats in the COUNT_STATS +updates the \*Count tables for each of the CountStats in the COUNT_STATS dictionary. The FillState table simply keeps track of the last end_time that we successfully updated each stat. It also enables the analytics system to recover from errors (by retrying) and to monitor that the cron job is @@ -103,7 +103,7 @@ There are a few important principles that we use to make the system efficient: - Not repeating work to keep things up to date (via FillState) -- Storing data in the *Count tables to avoid our endpoints hitting the core +- Storing data in the \*Count tables to avoid our endpoints hitting the core Message/UserMessage tables is key, because some queries could take minutes to calculate. This allows any expensive operations to run offline, and then the endpoints to server data to users can be fast. diff --git a/docs/subsystems/api-release-checklist.md b/docs/subsystems/api-release-checklist.md index 5dd5b8094c..d07c36a0a3 100644 --- a/docs/subsystems/api-release-checklist.md +++ b/docs/subsystems/api-release-checklist.md @@ -12,7 +12,7 @@ The steps below assume that you are familiar with the material presented [here](https://packaging.python.org/tutorials/installing-packages/). 1. [Reconfigure the package][2], if need be (upgrade version - number, development status, and so on). + number, development status, and so on). 2. Create a [source distribution][3]. diff --git a/docs/subsystems/billing.md b/docs/subsystems/billing.md index 70831b87d6..230a55eeda 100644 --- a/docs/subsystems/billing.md +++ b/docs/subsystems/billing.md @@ -4,6 +4,7 @@ Zulip uses a third party (Stripe) for billing, so working on the billing system requires a little bit of setup. To set up the development environment to work on the billing code: + - Create a Stripe account - Go to , and add the publishable key and secret key as `stripe_publishable_key` and @@ -15,6 +16,7 @@ Nearly all the billing-relevant code lives in `corporate/`. Stripe makes pretty regular updates to their API. The process for upgrading our code is: + - Go to in your Stripe account. - Upgrade the API version. - Run `tools/test-backend --generate-stripe-fixtures` diff --git a/docs/subsystems/caching.md b/docs/subsystems/caching.md index 497a57ec3a..a454b498eb 100644 --- a/docs/subsystems/caching.md +++ b/docs/subsystems/caching.md @@ -59,7 +59,7 @@ The `get_user` function is a pretty typical piece of code using this framework; as you can see, it's very little code on top of our `cache_with_key` decorator: -``` python +```python def user_profile_cache_key_id(email: str, realm_id: int) -> str: return u"user_profile:%s:%s" % (make_safe_digest(email.strip()), realm_id,) @@ -73,6 +73,7 @@ def get_user(email: str, realm: Realm) -> UserProfile: ``` This decorator implements a pretty classic caching paradigm: + - The `user_profile_cache_key` function defines a unique map from a canonical form of its arguments to a string. These strings are namespaced (the `user_profile:` part) so that they won't overlap @@ -237,6 +238,7 @@ objects to minimize data transfer between Django and memcached). We generally try to avoid in-process backend caching in Zulip's Django codebase, because every Zulip production installation involves multiple servers. We do have a few, however: + - `per_request_display_recipient_cache`: A cache flushed at the start of every request; this simplifies correctly implementing our goal of not repeatedly fetching the "display recipient" (e.g. stream name) @@ -260,5 +262,4 @@ cached by clients is changed. Clients are responsible for handling the events, updating their state, and rerendering any UI components that might display the modified state. -[post-save-signals]: -https://docs.djangoproject.com/en/2.0/ref/signals/#post-save +[post-save-signals]: https://docs.djangoproject.com/en/2.0/ref/signals/#post-save diff --git a/docs/subsystems/dependencies.md b/docs/subsystems/dependencies.md index 7599133e43..c7fccad35a 100644 --- a/docs/subsystems/dependencies.md +++ b/docs/subsystems/dependencies.md @@ -103,6 +103,7 @@ on specific versions of these packages wherever possible. The exact lists of `apt` packages needed by Zulip are maintained in a few places: + - For production, in our Puppet configuration, `puppet/zulip/`, using the `Package` and `SafePackage` directives. - For development, in `SYSTEM_DEPENDENCIES` in `tools/lib/provision.py`. @@ -249,12 +250,12 @@ installer for `yarn`, modified to support installing to a path that is not the current user's home directory). - `nvm` has its own system for installing each version of `node` at -its own path, which we use, though we install a `/usr/local/bin/node` -wrapper to access the desired version conveniently and efficiently -(`nvm` has a lot of startup overhead). + its own path, which we use, though we install a `/usr/local/bin/node` + wrapper to access the desired version conveniently and efficiently + (`nvm` has a lot of startup overhead). - `install-yarn.sh` is configured to install `yarn` at -`/srv/zulip-yarn`. We don't do anything special to try to manage -multiple versions of `yarn`. + `/srv/zulip-yarn`. We don't do anything special to try to manage + multiple versions of `yarn`. ## Other third-party and generated files diff --git a/docs/subsystems/email.md b/docs/subsystems/email.md index b36e9e9c80..6ff0a82d44 100644 --- a/docs/subsystems/email.md +++ b/docs/subsystems/email.md @@ -32,6 +32,7 @@ with only a few things you need to know to get started. One slightly complicated decision you may have to make when adding an email is figuring out how to schedule it. There are 3 ways to schedule email. + - Send it immediately, in the current Django process, e.g. by calling `send_email` directly. An example of this is the `confirm_registration` email. diff --git a/docs/subsystems/emoji.md b/docs/subsystems/emoji.md index 30fc440882..92204cb90a 100644 --- a/docs/subsystems/emoji.md +++ b/docs/subsystems/emoji.md @@ -68,6 +68,7 @@ emoji tooling). See [our dependencies document](../subsystems/dependencies.md) for more details on this strategy. The emoji tree generated by this process contains several import elements: + - `emoji_codes.json`: A set of mappings used by the Zulip frontend to understand what Unicode emoji exist and what their shortnames are, used for autocomplete, emoji pickers, etc. This has been diff --git a/docs/subsystems/full-text-search.md b/docs/subsystems/full-text-search.md index 18252a454e..6fab2cc525 100644 --- a/docs/subsystems/full-text-search.md +++ b/docs/subsystems/full-text-search.md @@ -63,8 +63,8 @@ All steps in this section should be run as the `root` user; on most installs, th su zulip -c '/home/zulip/deployments/current/manage.py migrate pgroonga' ``` - Note that the migration may take a long time, and users will be - unable to send new messages until the migration finishes. + Note that the migration may take a long time, and users will be + unable to send new messages until the migration finishes. 1. Once the migrations are complete, restart Zulip: @@ -80,9 +80,9 @@ All steps in this section should be run as the `root` user; on most installs, th su zulip -c '/home/zulip/deployments/current/manage.py migrate pgroonga zero' ``` - If you intend to re-enable PGroonga later, you can skip this step, - at the cost of your Message table being slightly larger than it would - be otherwise. + If you intend to re-enable PGroonga later, you can skip this step, + at the cost of your Message table being slightly larger than it would + be otherwise. 1. Edit `/etc/zulip/settings.py`, editing the line containing `USING_PGROONGA` to read: diff --git a/docs/subsystems/hashchange-system.md b/docs/subsystems/hashchange-system.md index 5e3e72acc5..0f2f60240e 100644 --- a/docs/subsystems/hashchange-system.md +++ b/docs/subsystems/hashchange-system.md @@ -14,7 +14,7 @@ Some examples are: "announce") selected. - `/#narrow/stream/42-android/topic/fun`: Message feed showing stream "android" and topic "fun". (The `42` represents the id of the - stream. + stream. The main module in the frontend that manages this all is `static/js/hashchange.js` (plus `hash_util.js` for all the parsing @@ -28,7 +28,7 @@ different flows: `/#streams`. This makes it easy to have simple links around the app without custom click handlers for each one. - The user uses the "back" button in their browser (basically - equivalent to the previous one, as a *link* out of the browser history + equivalent to the previous one, as a _link_ out of the browser history will be visited). - The user clicking some in-app click handler (e.g. "Stream settings" for an individual stream), that potentially does diff --git a/docs/subsystems/hotspots.md b/docs/subsystems/hotspots.md index 4e54abf366..c19b97bdd0 100644 --- a/docs/subsystems/hotspots.md +++ b/docs/subsystems/hotspots.md @@ -32,8 +32,8 @@ ALL_HOTSPOTS = { The target element and visual orientation of each hotspot is specified in `HOTSPOT_LOCATIONS` of `static/js/hotspots.js`. -The `icon_offset` property specifies where the pulsing icon is placed *relative to -the width and height of the target element*. +The `icon_offset` property specifies where the pulsing icon is placed _relative to +the width and height of the target element_. By default, `popovers.compute_placement` is used to responsively determine whether a popover is best displayed above (TOP), below (BOTTOM), @@ -53,6 +53,7 @@ in multiple copies of hotspots appearing; you can clear that by reloading the browser. Here are some visual characteristics to confirm: + - popover content is readable - icons reposition themselves on resize - icons are hidden and shown along with their associated elements @@ -67,6 +68,7 @@ a target element on a sidebar or overlay, the icon's z-index may need to be increased to 101, 102, or 103. This adjustment can be made at the bottom of `static/styles/hotspots.css`: + ```css #hotspot_new_hotspot_name_icon { z-index: 103; diff --git a/docs/subsystems/html-css.md b/docs/subsystems/html-css.md index 817412cf1a..26c18c3373 100644 --- a/docs/subsystems/html-css.md +++ b/docs/subsystems/html-css.md @@ -63,15 +63,15 @@ browsers to make sure things look the same. ### Behavior - Templates are automatically recompiled in development when the file -is saved; a refresh of the page should be enough to display the latest -version. You might need to do a hard refresh, as some browsers cache -webpages. + is saved; a refresh of the page should be enough to display the latest + version. You might need to do a hard refresh, as some browsers cache + webpages. - Variables can be used in templates. The variables available to the -template are called the **context**. Passing the context to the HTML -template sets the values of those variables to the value they were -given in the context. The sections below contain specifics on how the -context is defined and where it can be found. + template are called the **context**. Passing the context to the HTML + template sets the values of those variables to the value they were + given in the context. The sections below contain specifics on how the + context is defined and where it can be found. ### Backend templates @@ -85,11 +85,11 @@ found [here][jconditionals]. The context for Jinja2 templates is assembled from a few places: - `zulip_default_context` in `zerver/context_processors.py`. This is -the default context available to all Jinja2 templates. + the default context available to all Jinja2 templates. - As an argument in the `render` call in the relevant function that -renders the template. For example, if you want to find the context -passed to `index.html`, you can do: + renders the template. For example, if you want to find the context + passed to `index.html`, you can do: ```console $ git grep zerver/app/index.html '*.py' @@ -99,7 +99,7 @@ zerver/views/home.py: response = render(request, 'zerver/app/index.html', The next line in the code being the context definition. - `zproject/urls.py` for some fairly static pages that are rendered -using `TemplateView`, for example: + using `TemplateView`, for example: ```python path('config-error/google', TemplateView.as_view( @@ -224,9 +224,10 @@ If you want to test minified files in development, look for the ### How it works in production A few useful notes are: + - Zulip installs static assets in production in -`/home/zulip/prod-static`. When a new version is deployed, before the -server is restarted, files are copied into that directory. + `/home/zulip/prod-static`. When a new version is deployed, before the + server is restarted, files are copied into that directory. - We use the VFL (versioned file layout) strategy, where each file in the codebase (e.g. `favicon.ico`) gets a new name (e.g. `favicon.c55d45ae8c58.ico`) that contains a hash in it. Each @@ -281,8 +282,8 @@ production browser console. If you need to access a variable or function in those scenarios, add it to `zulip_test`. This is also **not** a stable API. -[Jinja2]: http://jinja.pocoo.org/ -[Handlebars]: https://handlebarsjs.com/ +[jinja2]: http://jinja.pocoo.org/ +[handlebars]: https://handlebarsjs.com/ [trans]: http://jinja.pocoo.org/docs/dev/templates/#i18n [jconditionals]: http://jinja.pocoo.org/docs/2.9/templates/#list-of-control-structures [hconditionals]: https://handlebarsjs.com/guide/#block_helpers.html diff --git a/docs/subsystems/input-pills.md b/docs/subsystems/input-pills.md index 344d49f842..08467a4125 100644 --- a/docs/subsystems/input-pills.md +++ b/docs/subsystems/input-pills.md @@ -66,7 +66,6 @@ export function filter_taken_users(items, pill_widget) { You can get notifications from the pill code that pills have been created/remove. - ```js pills.onPillCreate(function () { update_save_state(); diff --git a/docs/subsystems/logging.md b/docs/subsystems/logging.md index f450f54d08..9c1ae4d351 100644 --- a/docs/subsystems/logging.md +++ b/docs/subsystems/logging.md @@ -84,6 +84,7 @@ can be extremely valuable for investigating performance problems. ``` The format of this output is: + - Timestamp - Log level - Logger name, abbreviated as "zr" for these Zulip request logs @@ -92,11 +93,11 @@ The format of this output is: - HTTP status code - Time to process - (Optional perf data details, e.g. database time/queries, memcached -time/queries, Django process startup time, Markdown processing time, -etc.) + time/queries, Django process startup time, Markdown processing time, + etc.) - Endpoint/URL from zproject/urls.py - "email via client" showing user account involved (if logged in) and -the type of client they used ("web", "Android", etc.). + the type of client they used ("web", "Android", etc.). The performance data details are particularly useful for investigating performance problems, since one can see at a glance whether a slow @@ -136,6 +137,7 @@ new feature hard to miss. `blueslip = require("./static/js/blueslip"); blueslip.get_log()`. Blueslip supports several error levels: + - `throw new Error(…)`: For fatal errors that cannot be easily recovered from. We try to avoid using it, since it kills the current JS thread, rather than returning execution to the caller. diff --git a/docs/subsystems/management-commands.md b/docs/subsystems/management-commands.md index cac10977d4..f6a8371b95 100644 --- a/docs/subsystems/management-commands.md +++ b/docs/subsystems/management-commands.md @@ -21,7 +21,7 @@ written for a range of purposes: - The actual scripts run by supervisord to run the persistent processes in a Zulip server, e.g. `runtornado` and `process_queue`. - For a sysadmin to verify a Zulip server's configuration during -installation, e.g. `checkconfig`, `send_test_email`. + installation, e.g. `checkconfig`, `send_test_email`. - As the interface for doing those rare operations that don't have a UI yet, e.g. `deactivate_realm`, `reactivate_realm`, `change_user_email` (for the case where the user doesn't control the diff --git a/docs/subsystems/markdown.md b/docs/subsystems/markdown.md index 5fd01bd97c..22411ca72b 100644 --- a/docs/subsystems/markdown.md +++ b/docs/subsystems/markdown.md @@ -255,7 +255,6 @@ accurate. - Enable linking to other streams using `#**streamName**`. - ### Code - Enable fenced code block extension, with syntax highlighting. diff --git a/docs/subsystems/notifications.md b/docs/subsystems/notifications.md index 5fb596eab4..3148643ba7 100644 --- a/docs/subsystems/notifications.md +++ b/docs/subsystems/notifications.md @@ -25,24 +25,25 @@ sort of notifications system: As a reminder, the relevant part of the flow for sending messages is as follows: + - `do_send_messages` is the synchronous message-sending code path, and passing the following data in its `send_event` call: - Data about the message's content (E.g. mentions, wildcard - mentions, and alert words) and encodes it into the `UserMessage` - table's `flags` structure, which is in turn passed into - `send_event` for each user receiving the message. + mentions, and alert words) and encodes it into the `UserMessage` + table's `flags` structure, which is in turn passed into + `send_event` for each user receiving the message. - Data about user configuration relevant to the message, such as - `push_notify_user_ids` and `stream_notify_user_ids`, are included - alongside `flags` in the per-user data structure. + `push_notify_user_ids` and `stream_notify_user_ids`, are included + alongside `flags` in the per-user data structure. - The `presence_idle_user_ids` set, containing the subset of - recipient users who are mentioned, are PM recipients, have alert - words, or otherwise would normally get a notification, but have not - interacted with a Zulip client in the last few minutes. (Users who - have generally will not receive a notification unless the - `enable_online_push_notifications` flag is enabled). This data - structure ignores users for whom the message is not notifiable, - which is important to avoid this being thousands of `user_ids` for - messages to large streams with few currently active users. + recipient users who are mentioned, are PM recipients, have alert + words, or otherwise would normally get a notification, but have not + interacted with a Zulip client in the last few minutes. (Users who + have generally will not receive a notification unless the + `enable_online_push_notifications` flag is enabled). This data + structure ignores users for whom the message is not notifiable, + which is important to avoid this being thousands of `user_ids` for + messages to large streams with few currently active users. - The Tornado [event queue system](../subsystems/events-system.md) processes that data, as well as data about each user's active event queues, to (1) push an event to each queue needing that message and @@ -110,23 +111,23 @@ as follows: disabled are rechecked, as the user may have disabled one of these settings during the queuing period. - The **Email notifications queue processor**, `MissedMessageWorker`, - takes care to wait for 2 minutes (hopefully in the future this will be a - configuration setting) and starts a thread to batch together multiple - messages into a single email. These features are unnecessary - for mobile push notifications, because we can live-update those - details with a future notification, whereas emails cannot be readily - updated once sent. Zulip's email notifications are styled similarly - to GitHub's email notifications, with a clean, simple design that - makes replying from an email client possible (using the [incoming - email integration](../production/email-gateway.md)). + takes care to wait for 2 minutes (hopefully in the future this will be a + configuration setting) and starts a thread to batch together multiple + messages into a single email. These features are unnecessary + for mobile push notifications, because we can live-update those + details with a future notification, whereas emails cannot be readily + updated once sent. Zulip's email notifications are styled similarly + to GitHub's email notifications, with a clean, simple design that + makes replying from an email client possible (using the [incoming + email integration](../production/email-gateway.md)). - The **Push notifications queue processor**, - `PushNotificationsWorker`, is a simple wrapper around the - `push_notifications.py` code that actually sends the - notification. This logic is somewhat complicated by having to track - the number of unread push notifications to display on the mobile - apps' badges, as well as using the [mobile push notifications - service](../production/mobile-push-notifications.md) for self-hosted - systems. + `PushNotificationsWorker`, is a simple wrapper around the + `push_notifications.py` code that actually sends the + notification. This logic is somewhat complicated by having to track + the number of unread push notifications to display on the mobile + apps' badges, as well as using the [mobile push notifications + service](../production/mobile-push-notifications.md) for self-hosted + systems. The following important constraints are worth understanding about the structure of the system, when thinking about changes to it: diff --git a/docs/subsystems/performance.md b/docs/subsystems/performance.md index da10697369..d5a868f715 100644 --- a/docs/subsystems/performance.md +++ b/docs/subsystems/performance.md @@ -29,6 +29,7 @@ important to understand the load profiles for production uses. Zulip servers typically involve a mixture of two very different types of load profiles: + - Open communities like open source projects, online classes, etc. have large numbers of users, many of whom are idle. (Many of the others likely stopped by to ask a question, got it answered, and @@ -83,7 +84,7 @@ substantial oscillation within a 24 hour period), we expect the rough sense of them (as well as the list of important endpoints) is not likely to vary dramatically over time. -``` eval_rst +```eval_rst ======================= ============ ============== =============== Endpoint Average time Request volume Average impact ======================= ============ ============== =============== @@ -181,7 +182,7 @@ Zulip is somewhat unusual among webapps in sending essentially all of the data required for the entire Zulip webapp in this single request, which is part of why the Zulip webapp loads very quickly -- one only needs a single round trip aside from cacheable assets (avatars, images, JS, -CSS). Data on other users in the organization, streams, supported +CSS). Data on other users in the organization, streams, supported emoji, custom profile fields, etc., is all included. The nice thing about this model is that essentially every UI element in the Zulip client can be rendered immediately without paying latency to the diff --git a/docs/subsystems/pointer.md b/docs/subsystems/pointer.md index d8b77fe5be..6e3830f7c2 100644 --- a/docs/subsystems/pointer.md +++ b/docs/subsystems/pointer.md @@ -28,7 +28,7 @@ First a bit of terminology: ### Recipient bar: message you clicked -If you enter a narrow by clicking on a message group's *recipient bar* +If you enter a narrow by clicking on a message group's _recipient bar_ (stream/topic or private message recipient list at the top of a group of messages), Zulip will select the message you clicked on. This provides a nice user experience where you get to see the stuff near diff --git a/docs/subsystems/realms.md b/docs/subsystems/realms.md index 8ea8b90ccf..4f442dcf45 100644 --- a/docs/subsystems/realms.md +++ b/docs/subsystems/realms.md @@ -1,6 +1,6 @@ # Realms in Zulip -Zulip allows multiple *realms* to be hosted on a single instance. +Zulip allows multiple _realms_ to be hosted on a single instance. Realms are the Zulip codebases's internal name for what we refer to in user documentation as an organization (the name "realm" comes from [Kerberos](https://web.mit.edu/kerberos/)). @@ -66,7 +66,7 @@ user-facing documentation on this. By default, Linux does not provide a convenient way to use subdomains in your local development environment. To solve this problem, we use the **zulipdev.com** domain, which has a wildcard A record pointing to -127.0.0.1. You can use zulipdev.com to connect to your Zulip +127.0.0.1. You can use zulipdev.com to connect to your Zulip development server instead of localhost. The default realm with the Shakespeare users has the subdomain `zulip` and can be accessed by visiting **zulip.zulipdev.com**. @@ -77,7 +77,7 @@ will try to get the page on your behalf. Since zulipdev.com points to 127.0.0.1 the proxy server is likely to give you a 503 error. The workaround is to disable your proxy for `*.zulipdev.com`. The DNS lookup should still work even if you disable proxy for -*.zulipdev.com. If it doesn't you can add zulipdev.com records in +\*.zulipdev.com. If it doesn't you can add zulipdev.com records in `/etc/hosts` file. The file should look something like this. ```text diff --git a/docs/subsystems/release-checklist.md b/docs/subsystems/release-checklist.md index 224a4ec4e1..68f41fd3db 100644 --- a/docs/subsystems/release-checklist.md +++ b/docs/subsystems/release-checklist.md @@ -65,6 +65,7 @@ preparing a new release. **Note:** This will trigger the [GitHub action](https://github.com/zulip/zulip/blob/main/tools/oneclickapps/README.md) for updating DigitalOcean one-click app image. The action uses the latest release tarball published on `zulip.org` for creating the image. + - Update the [Docker image](https://github.com/zulip/docker-zulip) and do a release of that. - Update the image of DigitalOcean one click app using diff --git a/docs/subsystems/schema-migrations.md b/docs/subsystems/schema-migrations.md index 8ec72fa2c0..b8aee2adb9 100644 --- a/docs/subsystems/schema-migrations.md +++ b/docs/subsystems/schema-migrations.md @@ -93,16 +93,18 @@ migrations. Another important note is that making changes to the data in a table via `RunPython` code and `ALTER TABLE` operations within a single, atomic migration don't mix well. If you encounter an error such as + ```text django.db.utils.OperationalError: cannot ALTER TABLE "table_name" because it has pending trigger events ``` + when testing the migration, the reason is often that these operations were incorrectly mixed. To resolve this, consider making the migration non-atomic, splitting it into two migration files (recommended), or replacing the `RunPython` logic with pure SQL (though this can generally be difficult). - **Making large migrations work**. Major migrations should have a -few properties: + few properties: - **Unit tests**. You'll want to carefully test these, so you might as well write some unit tests to verify the migration works @@ -120,13 +122,13 @@ few properties: the migration can even continue where it left off, without needing to redo work. - **Multi-step migrations**. For really big migrations, one wants - to split the transition into into several commits that are each - individually correct, and can each be deployed independently: + to split the transition into into several commits that are each + individually correct, and can each be deployed independently: 1. First, do a migration to add the new column to the Message table - and start writing to that column (but don't use it for anything) + and start writing to that column (but don't use it for anything) 2. Second, do a migration to copy values from the old column to - the new column, to ensure that the two data stores agree. + the new column, to ensure that the two data stores agree. 3. Third, a commit that stops writing to the old field. 4. Any cleanup work, e.g. if the old field were a column, we'd do a migration to remove it entirely here. diff --git a/docs/subsystems/sending-messages.md b/docs/subsystems/sending-messages.md index 4a52f31e5d..a4a2cc2215 100644 --- a/docs/subsystems/sending-messages.md +++ b/docs/subsystems/sending-messages.md @@ -17,16 +17,17 @@ and we generally don't repeat the content discussed there. This is just a bit of terminology: A "message list" is what Zulip calls the frontend concept of a (potentially narrowed) message feed. There are 3 related structures: + - A `message_list_data` just has the sequencing data of which message -IDs go in what order. + IDs go in what order. - A `message_list` is built on top of `message_list_data` and -additionally contains the data for a visible-to-the-user message list -(E.g. where trailing bookends should appear, a selected message, -etc.). + additionally contains the data for a visible-to-the-user message list + (E.g. where trailing bookends should appear, a selected message, + etc.). - A `message_list_view` is built on top of `message_list` and -additionally contains rendering details like a window of up to 400 -messages that is present in the DOM at the time, scroll position -controls, etc. + additionally contains rendering details like a window of up to 400 + messages that is present in the DOM at the time, scroll position + controls, etc. (This should later be expanded into a full article on message lists and narrowing). @@ -49,26 +50,26 @@ process described in our This section details the ways in which it is different: - There is significant custom code inside the `process_message_event` -function in `zerver/tornado/event_queue.py`. This custom code has a -number of purposes: - - Triggering [email and mobile push - notifications](../subsystems/notifications.md) for any users who - do not have active clients and have settings of the form "push - notifications when offline". In order to avoid doing any real - computational work inside the Tornado codebase, this logic aims - to just do the check for whether a notification should be - generated, and then put an event into an appropriate - [queue](../subsystems/queuing.md) to actually send the message. - See `maybe_enqueue_notifications` and related code for this part - of the logic. - - Splicing user-dependent data (E.g. `flags` such as when the user - was `mentioned`) into the events. - - Handling the [local echo details](#local-echo). - - Handling certain client configuration options that affect - messages. E.g. determining whether to send the - plaintext/Markdown raw content or the rendered HTML (e.g. the - `apply_markdown` and `client_gravatar` features in our - [events API docs](https://zulip.com/api/register-queue)). + function in `zerver/tornado/event_queue.py`. This custom code has a + number of purposes: + - Triggering [email and mobile push + notifications](../subsystems/notifications.md) for any users who + do not have active clients and have settings of the form "push + notifications when offline". In order to avoid doing any real + computational work inside the Tornado codebase, this logic aims + to just do the check for whether a notification should be + generated, and then put an event into an appropriate + [queue](../subsystems/queuing.md) to actually send the message. + See `maybe_enqueue_notifications` and related code for this part + of the logic. + - Splicing user-dependent data (E.g. `flags` such as when the user + was `mentioned`) into the events. + - Handling the [local echo details](#local-echo). + - Handling certain client configuration options that affect + messages. E.g. determining whether to send the + plaintext/Markdown raw content or the rendered HTML (e.g. the + `apply_markdown` and `client_gravatar` features in our + [events API docs](https://zulip.com/api/register-queue)). - Following our standard naming convention, input validation is done inside the `check_message` function in `zerver/lib/actions.py`, which is responsible for validating the user can send to the recipient, @@ -78,27 +79,27 @@ number of purposes: the message) in `zerver/lib/actions.py` is one of the most optimized and thus complex parts of the system. But in short, its job is to atomically do a few key things: - - Store a `Message` row in the database. - - Store one `UserMessage` row in the database for each user who is - a recipient of the message (including the sender), with - appropriate `flags` for whether the user was mentioned, an alert - word appears, etc. See - [the section on soft deactivation](#soft-deactivation) for - a clever optimization we use here that is important for large - open organizations. - - Do all the database queries to fetch relevant data for and then - send a `message` event to the - [events system](../subsystems/events-system.md) containing the - data it will need for the calculations described above. This - step adds a lot of complexity, because the events system cannot - make queries to the database directly. - - Trigger any other deferred work caused by the current message, - e.g. [outgoing webhooks](https://zulip.com/api/outgoing-webhooks) - or embedded bots. - - Every query is designed to be a bulk query; we carefully - unit-test this system for how many database and memcached queries - it makes when sending messages with large numbers of recipients, - to ensure its performance. + - Store a `Message` row in the database. + - Store one `UserMessage` row in the database for each user who is + a recipient of the message (including the sender), with + appropriate `flags` for whether the user was mentioned, an alert + word appears, etc. See + [the section on soft deactivation](#soft-deactivation) for + a clever optimization we use here that is important for large + open organizations. + - Do all the database queries to fetch relevant data for and then + send a `message` event to the + [events system](../subsystems/events-system.md) containing the + data it will need for the calculations described above. This + step adds a lot of complexity, because the events system cannot + make queries to the database directly. + - Trigger any other deferred work caused by the current message, + e.g. [outgoing webhooks](https://zulip.com/api/outgoing-webhooks) + or embedded bots. + - Every query is designed to be a bulk query; we carefully + unit-test this system for how many database and memcached queries + it makes when sending messages with large numbers of recipients, + to ensure its performance. ## Local echo @@ -176,6 +177,7 @@ implementation was under 150 lines of code. This section just has a brief review of the sequence of steps all in one place: + - User hits send in the compose box. - Compose box validation runs; if it passes, the browser locally echoes the message and then sends a request to the `POST /messages` @@ -183,7 +185,7 @@ one place: - The Django URL routes and middleware run, and eventually call the `send_message_backend` view function in `zerver/views/messages.py`. (Alternatively, for an API request to send a message via Zulip's - REST API, things start here). + REST API, things start here). - `send_message_backend` does some validation before triggering the `check_message` + `do_send_messages` backend flow. - That backend flow saves the data to the database and triggers a @@ -233,17 +235,17 @@ significant delay in rendering the message and delivering it to other users. - For this case, Zulip's backend Markdown processor will render the -message without including the URL embeds/previews, but it will add a -deferred work item into the `embed_links` queue. + message without including the URL embeds/previews, but it will add a + deferred work item into the `embed_links` queue. - The [queue processor](../subsystems/queuing.md) for the -`embed_links` queue will fetch the URLs, and then if they return -results, rerun the Markdown processor and notify clients of the -updated message `rendered_content`. + `embed_links` queue will fetch the URLs, and then if they return + results, rerun the Markdown processor and notify clients of the + updated message `rendered_content`. - We reuse the `update_message` framework (used for -Zulip's message editing feature) in order to avoid needing custom code -to implement the notification-and-rerender part of this implementation. + Zulip's message editing feature) in order to avoid needing custom code + to implement the notification-and-rerender part of this implementation. ## Soft deactivation @@ -328,43 +330,43 @@ organization for a few weeks, they are tagged as soft-deactivated. The way this works internally is: - We (usually) skip creating UserMessage rows for soft-deactivated -users when a message is sent to a stream where they are subscribed. + users when a message is sent to a stream where they are subscribed. - If/when the user ever returns to Zulip, we can at that time -reconstruct the UserMessage rows that they missed, and create the rows -at that time (or, to avoid a latency spike if/when the user returns to -Zulip, this work can be done in a nightly cron job). We can construct -those rows later because we already have the data for when the user -might have been subscribed or unsubscribed from streams by other -users, and, importantly, we also know that the user didn’t interact -with the UI since the message was sent (and thus we can safely assume -that the messages have not been marked as read by the user). This is -done in the `add_missing_messages` function, which is the core of the -soft-deactivation implementation. + reconstruct the UserMessage rows that they missed, and create the rows + at that time (or, to avoid a latency spike if/when the user returns to + Zulip, this work can be done in a nightly cron job). We can construct + those rows later because we already have the data for when the user + might have been subscribed or unsubscribed from streams by other + users, and, importantly, we also know that the user didn’t interact + with the UI since the message was sent (and thus we can safely assume + that the messages have not been marked as read by the user). This is + done in the `add_missing_messages` function, which is the core of the + soft-deactivation implementation. - The “usually” above is because there are a few flags that result -from content in the message (e.g., a message that mentions a user -results in a “mentioned” flag in the UserMessage row), that we need to -keep track of. Since parsing a message can be expensive (>10ms of -work, depending on message content), it would be too inefficient to -need to re-parse every message when a soft-deactivated user comes back -to Zulip. Conveniently, those messages are rare, and so we can just -create UserMessage rows which would have “interesting” flags at the -time they were sent without any material performance impact. And then -`add_missing_messages` skips any messages that already have a -`UserMessage` row for that user when doing its backfill. + from content in the message (e.g., a message that mentions a user + results in a “mentioned” flag in the UserMessage row), that we need to + keep track of. Since parsing a message can be expensive (>10ms of + work, depending on message content), it would be too inefficient to + need to re-parse every message when a soft-deactivated user comes back + to Zulip. Conveniently, those messages are rare, and so we can just + create UserMessage rows which would have “interesting” flags at the + time they were sent without any material performance impact. And then + `add_missing_messages` skips any messages that already have a + `UserMessage` row for that user when doing its backfill. The end result is the best of both worlds: - Nobody's view of the world is different because the user was -soft-deactivated (resulting in no visible user-experience impact), at -least if one is running the cron job. If one does not run the cron -job, then users returning after being away for a very long time will -potentially have a (very) slow loading experience as potentially -100,000s of UserMessage rows might need to be reconstructed at once. + soft-deactivated (resulting in no visible user-experience impact), at + least if one is running the cron job. If one does not run the cron + job, then users returning after being away for a very long time will + potentially have a (very) slow loading experience as potentially + 100,000s of UserMessage rows might need to be reconstructed at once. - On the latency-sensitive message sending and fanout code path, the -server only needs to do work for users who are currently interacting -with Zulip. + server only needs to do work for users who are currently interacting + with Zulip. Empirically, we've found this technique completely resolved the "send latency" scaling problem. The latency of sending a message to a stream @@ -374,6 +376,7 @@ it’ll arrive in the couple hundred milliseconds one would expect if the extra 4500 inactive subscribers didn’t exist. There are a few details that require special care with this system: + - [Email and mobile push notifications](../subsystems/notifications.md). We need to make sure these are still correctly delivered to soft-deactivated users; diff --git a/docs/subsystems/settings.md b/docs/subsystems/settings.md index e35379d2c4..24c0c660c3 100644 --- a/docs/subsystems/settings.md +++ b/docs/subsystems/settings.md @@ -5,6 +5,7 @@ help you decide how to correctly implement new settings you're adding to Zulip. We have two types of administrative settings in Zulip: + - **Server settings** are set via configuration files, and apply to the whole Zulip installation. - **Realm settings** (or **organization settings**) are usually @@ -16,11 +17,11 @@ Philosophically, the goals of the settings system are to make it convenient for: - Zulip server administrators to configure -Zulip's featureset for their server without needing to patch Zulip + Zulip's featureset for their server without needing to patch Zulip - Realm administrators to configure settings for their organization -independently without needing to talk with the server administrator. + independently without needing to talk with the server administrator. - Secrets (passwords, API keys, etc.) to be stored in a separate place -from shareable configuration. + from shareable configuration. ## Server settings diff --git a/docs/subsystems/widgets.md b/docs/subsystems/widgets.md index 58a772b531..97f40e8035 100644 --- a/docs/subsystems/widgets.md +++ b/docs/subsystems/widgets.md @@ -249,7 +249,7 @@ inside a `choices` list inside of the JSON payload. Here is what an example payload looks like: -~~~ json +```json { "extra_data": { "type": "choices", @@ -283,7 +283,7 @@ Here is what an example payload looks like: }, "widget_type": "zform" } -~~~ +``` When users click on the buttons, **generic** click handlers automatically simulate a client reply using @@ -301,12 +301,11 @@ are completely generic. We can walk through the steps from the bot generating the **zform** to the client rendering it. - First, [here](https://github.com/zulip/python-zulip-api/blob/main/zulip_bots/zulip_bots/bots/trivia_quiz/trivia_quiz.py) is the code that produces the JSON. -``` py +```py def format_quiz_for_widget(quiz_id: str, quiz: Dict[str, Any]) -> str: widget_type = 'zform' question = quiz['question'] @@ -366,16 +365,16 @@ sibling of **poll** and **zform** just has a somewhat more generic job to do.) In `static/js/widgetize.js` you will see where this code converges, with snippets like this: -~~~ js +```js widgets.poll = poll_widget; widgets.todo = todo_widget; widgets.zform = zform; -~~~ +``` The code in `static/js/zform.js` renders the form (not shown here) and then sets up a click handler like below: -~~~ js +```js elem.find('button').on('click', function (e) { e.stopPropagation(); @@ -391,7 +390,7 @@ shown here) and then sets up a click handler like below: content: reply_content, }); }); -~~~ +``` And then we are basically done! diff --git a/docs/testing/continuous-integration.md b/docs/testing/continuous-integration.md index f7ca8f63dd..3d537a83bc 100644 --- a/docs/testing/continuous-integration.md +++ b/docs/testing/continuous-integration.md @@ -14,34 +14,34 @@ and false positives, both of which can waste a lot of developer time. There are a few implications of this overall goal: - If a test is failing nondeterministically in CI, we consider that to -be an urgent problem. + be an urgent problem. - If the tests become a lot slower, that is also an urgent problem. - Everything we do in CI should also have a way to run it quickly -(under 1 minute, preferably under 3 seconds), in order to iterate fast -in development. Except when working on the CI configuration itself, a -developer should never have to repeatedly wait 10 minutes for a full CI -run to iteratively debug something. + (under 1 minute, preferably under 3 seconds), in order to iterate fast + in development. Except when working on the CI configuration itself, a + developer should never have to repeatedly wait 10 minutes for a full CI + run to iteratively debug something. ## GitHub Actions ### Useful debugging tips and tools - GitHub Actions stores timestamps for every line in the logs. They -are hidden by default; you can see them by toggling the -`Show timestamps` option in the menu on any job's log page. (You can -get this sort of timestamp in a development environment by piping -output to `ts`). + are hidden by default; you can see them by toggling the + `Show timestamps` option in the menu on any job's log page. (You can + get this sort of timestamp in a development environment by piping + output to `ts`). - GitHub Actions runs on every branch you push on your Zulip fork. -This is helpful when debugging something complicated. + This is helpful when debugging something complicated. - You can also ssh into a container to debug failures. SSHing into -the containers can be helpful, especially in rare cases where the -tests are passing in your computer but failing in the CI. There are -various -[Actions](https://github.com/marketplace?type=actions&query=debug+ssh) -available on GitHub Marketplace to help you SSH into a container. Use -whichever you find easiest to set up. + the containers can be helpful, especially in rare cases where the + tests are passing in your computer but failing in the CI. There are + various + [Actions](https://github.com/marketplace?type=actions&query=debug+ssh) + available on GitHub Marketplace to help you SSH into a container. Use + whichever you find easiest to set up. ### Suites diff --git a/docs/testing/linters.md b/docs/testing/linters.md index b9bf5a5683..9c0412be8e 100644 --- a/docs/testing/linters.md +++ b/docs/testing/linters.md @@ -46,6 +46,7 @@ You can also run them individually or pass specific files: `./tools/lint` has many useful options; you can read about them in its internal documentation using `./tools/lint --help`. Of particular note are: + - `--fix`: Several of our linters support automatically fixing basic issues; this option will ask `tools/lint` to run those. - `--verbose`: Provides detailed information on how to fix many common @@ -191,6 +192,7 @@ that we exempt may be deemed not worthwhile to fix. #### JavaScript code We check our JavaScript code in a few different ways: + - We run eslint. - We check code formatting with Prettier. - We perform custom Zulip regex checks on the code. diff --git a/docs/testing/manual-testing.md b/docs/testing/manual-testing.md index 01f1f43617..17e7f66dcb 100644 --- a/docs/testing/manual-testing.md +++ b/docs/testing/manual-testing.md @@ -1,4 +1,4 @@ -# Manual testing # +# Manual testing As a general rule, we like to have automated tests for everything that can be practically tested. However, there are certain types of bugs @@ -11,7 +11,7 @@ This doc assumes you know how to set up a local development server and open the Zulip app in the browser. It also assumes a basic knowledge of how to use Zulip. -## Basic stuff ## +## Basic stuff When testing Zulip manually, here are things to focus on: @@ -36,12 +36,13 @@ The rest of this document groups tasks into basic areas of functionality of the system. If you have multiple people testing at once, you can divvy up QA tasks by these sections in the doc. -### Message view ### +### Message view We mostly test the message view as part of testing everything else, but there are few things to specially test here. Try using all the navigation hotkeys: + - Up/k - Down/j - PgUp/K @@ -50,103 +51,107 @@ Try using all the navigation hotkeys: - also try scrolling aggressively with the mouse Try narrowing from the message view: -- Hotkeys - - use a to go to All messages - - use s to narrow to a stream (select message first - and verify in sidebar) - - use S to narrow to the topic (and verify in sidebar) - - use v to navigate to private messages -- Click on the recipient bar - - narrow to a stream - - narrow to a topic - - narrow to PMs with one user - - narrow to a group PM -- Click on the Zulip logo - - narrow to a topic - - click on the Zulip logo (and verify you're in the Recent topics view) -### Messagebox ### +- Hotkeys + - use a to go to All messages + - use s to narrow to a stream (select message first + and verify in sidebar) + - use S to narrow to the topic (and verify in sidebar) + - use v to navigate to private messages +- Click on the recipient bar + - narrow to a stream + - narrow to a topic + - narrow to PMs with one user + - narrow to a group PM +- Click on the Zulip logo + - narrow to a topic + - click on the Zulip logo (and verify you're in the Recent topics view) + +### Messagebox With messagebox we want to test mainly the functioning of all all the keyboard shortcuts and click handlers. Apart from that there are three views of a message box, we want to test their appearance too: + - Message that includes sender - Message without the sender - "/me" message Here's how we're going to test the message appearances: + - narrow to a new topic and send a message (this message will include sender) - - edit the message ("(EDITED)" label should appear beside sender name) + - edit the message ("(EDITED)" label should appear beside sender name) - send another message (will not include sender) - - edit the message ("(EDITED)" label should appear in the left column, where the avatar is) + - edit the message ("(EDITED)" label should appear in the left column, where the avatar is) - send a "/me" message (`/me test message`) - - message should appear alongside sender name - - edit the message ("(EDITED)" label should appear beside the message) + - message should appear alongside sender name + - edit the message ("(EDITED)" label should appear beside the message) For all the three cases, we need to test the click handlers and the hotkeys too: + - Sender popover: - - click on the avatar and sender name for messages which include sender - - press 'u' to open the sender popover for all messages + - click on the avatar and sender name for messages which include sender + - press 'u' to open the sender popover for all messages - Message reply: - - click on message to reply - - use 'r' or Return hotkey to reply - - use '>' to quote and reply - - use '@' to mention and reply + - click on message to reply + - use 'r' or Return hotkey to reply + - use '>' to quote and reply + - use '@' to mention and reply - Reactions: - - click on the reactions button to open menu - - use ':' to open the reactions menu - - react to a message + - click on the reactions button to open menu + - use ':' to open the reactions menu + - react to a message - Chevron - - click on chevron to open menu - - use 'i' to open chevron menu + - click on chevron to open menu + - use 'i' to open chevron menu - Message edit - - click on message edit/view source button - - use 'i' + Return to edit/view source message - - click on the 'copy and close' option in view source and verify positioning of 'Copied!' label. + - click on message edit/view source button + - use 'i' + Return to edit/view source message + - click on the 'copy and close' option in view source and verify positioning of 'Copied!' label. - Star a message: - - click on the star button in the right column - - use 'Ctrl + S' to star a message + - click on the star button in the right column + - use 'Ctrl + S' to star a message - Message length - - send a long message and see if '[More]' appears - - click on the 'more' or 'collapse' link - - use i to collapse/expand a message irrespective of message length + - send a long message and see if '[More]' appears + - click on the 'more' or 'collapse' link + - use i to collapse/expand a message irrespective of message length - use 'v' to show all images in the thread - use 'M' to mute the thread Play with the screen size to check if the messagebox appears fine in different screens too. -### Message editing ### +### Message editing With message editing we mainly want to exercise topic changes. Here are some tasks: - Do lots of editing - - send a message to the topic "original" - - edit the message content - - send two messages to the "original" stream - - start to edit a message but then cancel - - change the topic for the first message to "change1" (just - this message) - - narrow back to "original" - - send one more message to the stream - - change the topic for the last two messages to "change2" - - narrow back to "original" - - send two more messages to the stream - - edit the 2nd message on topic and change all messages to - "change3" + + - send a message to the topic "original" + - edit the message content + - send two messages to the "original" stream + - start to edit a message but then cancel + - change the topic for the first message to "change1" (just + this message) + - narrow back to "original" + - send one more message to the stream + - change the topic for the last two messages to "change2" + - narrow back to "original" + - send two more messages to the stream + - edit the 2nd message on topic and change all messages to + "change3" - Test UI entry points - - hit "i" then down arrow to edit with the popup - - use the popup using the mouse - - enter edit mode using the pencil icon + - hit "i" then down arrow to edit with the popup + - use the popup using the mouse + - enter edit mode using the pencil icon - -### Narrowing ### +### Narrowing Zulip uses the term "narrowing" to refer to opening different views of your messages, whether by clicking on sidebar options, recipient @@ -155,7 +160,6 @@ be watching unread counts. Of course, you also want to see messages show up in the message pane. And, finally, you should make sure that no messages outside the narrow show up in Cordelia's view. - ```eval_rst .. important:: Make sure that Cordelia is subscribed to Verona but not @@ -195,7 +199,7 @@ There are 56 things to test here. If you can get into a rhythm where you can test each case in about 30 seconds, then the whole exercise is about 30 minutes, assuming no bugs. -### Composing messages ### +### Composing messages We have pretty good automated tests for our Markdown processor, so manual testing is targeted more to other interactions. For composing @@ -203,59 +207,65 @@ a message, pay attention to details like what is automatically populated and where the focus is placed. - Hotkeys - - use r to reply to a stream message - - use r to reply to a PM - - use R to reply to the author of a PM - - use R to reply to the author of a PM stream - - use c to compose a stream message - - use x to compose a new PM + + - use r to reply to a stream message + - use r to reply to a PM + - use R to reply to the author of a PM + - use R to reply to the author of a PM stream + - use c to compose a stream message + - use x to compose a new PM - Buttons - - Narrow to a stream and click on "New topic" - - Narrow "Private messages" and click on "New topic" - - Narrow to a stream and click on "New private message" - - Narrow "Private messages" and click on "New private message" + + - Narrow to a stream and click on "New topic" + - Narrow "Private messages" and click on "New topic" + - Narrow to a stream and click on "New private message" + - Narrow "Private messages" and click on "New private message" - Topics - - Compose/send a message to a stream with no topic. - - Compose/send a message to a stream with a new topic. - - Compose/send a message to a stream with autocomplete. - - Compose/send a message to a stream manually typing an - existing topic. + + - Compose/send a message to a stream with no topic. + - Compose/send a message to a stream with a new topic. + - Compose/send a message to a stream with autocomplete. + - Compose/send a message to a stream manually typing an + existing topic. - Formatting stuff - - Use the "A" icon to get Markdown help. - - Use the eyeball icon to show a preview and send from preview mode. - - Toggle in and out of preview before sending a message. - - Use @-mention to mention Hamlet (and send him a message). - - Use `#**devel**` syntax and send to Hamlet, then follow the link. - - Create a bulleted list. - - Use the emoji icon to find an emoji in the picker. + + - Use the "A" icon to get Markdown help. + - Use the eyeball icon to show a preview and send from preview mode. + - Toggle in and out of preview before sending a message. + - Use @-mention to mention Hamlet (and send him a message). + - Use `#**devel**` syntax and send to Hamlet, then follow the link. + - Create a bulleted list. + - Use the emoji icon to find an emoji in the picker. - Attachments - - Send a message with an attachment using the paperclip icon. - - Send a message with multiple attachments. - - Copy an image from the clipboard. - - Use drag/drop from the desktop to upload an image. + + - Send a message with an attachment using the paperclip icon. + - Send a message with multiple attachments. + - Copy an image from the clipboard. + - Use drag/drop from the desktop to upload an image. - Drafts - - Start composing a message then click outside the compose box. - - Use "restore drafts" to restore the draft. - - Start composing then use "Esc" to abort the message. - - Use "restore drafts" to restore the draft. - - Start composing a stream message and then abort using - the little "x" icon in the compose box. - - Click on "New private message" and restore the draft. (You - should now be sending to a stream.) + + - Start composing a message then click outside the compose box. + - Use "restore drafts" to restore the draft. + - Start composing then use "Esc" to abort the message. + - Use "restore drafts" to restore the draft. + - Start composing a stream message and then abort using + the little "x" icon in the compose box. + - Click on "New private message" and restore the draft. (You + should now be sending to a stream.) - Click to send - - Turn off Enter-to-send. - - Send a two-paragraph message using Tab and Enter. - - Send a two-paragraph message using Ctrl-Enter or Cmd-Enter. - - Turn on Enter-to-send. - - Hit Enter to send. + - Turn off Enter-to-send. + - Send a two-paragraph message using Tab and Enter. + - Send a two-paragraph message using Ctrl-Enter or Cmd-Enter. + - Turn on Enter-to-send. + - Hit Enter to send. -### Popover menus ### +### Popover menus For this task you just want to go through all of our popover menus and exercise them. The main nuance here is that you occasionally want @@ -267,66 +277,70 @@ then the message will disappear from the view. Here are the things to test: - Stream sidebar menus (click ellipsis when hovering over stream filters) - - Stream settings (just make sure it goes there) - - Narrow (and then have Hamlet send a message) - - Pin/unpin (do both) - - Compose (send a message to the stream) - - Mark as read (scroll back and then have Hamlet send you a message) - - Mute/unmute (do both) - - Unsubscribe (and then go to Stream settings in the gear menu to resubscribe) - - Choose custom color (play around with this) + + - Stream settings (just make sure it goes there) + - Narrow (and then have Hamlet send a message) + - Pin/unpin (do both) + - Compose (send a message to the stream) + - Mark as read (scroll back and then have Hamlet send you a message) + - Mute/unmute (do both) + - Unsubscribe (and then go to Stream settings in the gear menu to resubscribe) + - Choose custom color (play around with this) - Topic sidebar menus (click ellipsis when hovering over topics) - - Narrow (and then have Hamlet send a message) - - Mute/unmute (try both) - - Mark as read (scroll back and then have Hamlet send you a message) + + - Narrow (and then have Hamlet send a message) + - Mute/unmute (try both) + - Mark as read (scroll back and then have Hamlet send you a message) - Left-message-pane menus (click on person's name) - - Verify email - - Verify date message sent - - Send a PM (make sure compose box is filled out ok) - - Narrow to PMs with - - Narrow to PMs sent by + + - Verify email + - Verify date message sent + - Send a PM (make sure compose box is filled out ok) + - Narrow to PMs with + - Narrow to PMs sent by - Right-pane-pane menus (click on chevron when hovering) - - use "i" hotkey to open the menu - - Edit a message you sent (using the down-arrow key to navigate the popup) - - View Source for somebody else's message (make sure - it's not editable) - - Reply (send a message) - - Collapse/uncollapse (try both) - - Mute/unmute (try both, watch left sidebar) - - Link to this conversation + + - use "i" hotkey to open the menu + - Edit a message you sent (using the down-arrow key to navigate the popup) + - View Source for somebody else's message (make sure + it's not editable) + - Reply (send a message) + - Collapse/uncollapse (try both) + - Mute/unmute (try both, watch left sidebar) + - Link to this conversation - Buddy list menus (click ellipsis when hovering over users) - - Narrow to PMs with - - Narrow to message sent by - - Compose a message to + - Narrow to PMs with + - Narrow to message sent by + - Compose a message to - -### Sidebar filtering ### +### Sidebar filtering This is a fairly quick task where we test the search filters on the left sidebar and the buddy list. If Cordelia is not subscribed to Denmark, subscribe her to that stream. - Streams filtering - - Use "w" hotkey to open the search. - - Filter on "d". - - Pin/unpin Denmark. - - Clear filter. - - Use "A" and "D" hotkeys to cycle through the streams. - - Filter again and then click somewhere else. + + - Use "w" hotkey to open the search. + - Filter on "d". + - Pin/unpin Denmark. + - Clear filter. + - Use "A" and "D" hotkeys to cycle through the streams. + - Filter again and then click somewhere else. - Buddy list filtering - - Use "q" hotkey to open the search. - - Filter for Hamlet, Prospero, Othello, etc. - - Log on Hamlet and log off Hamlet while filtering for Hamlet. - - Log on/log off Hamlet while filtering for Othello. - - Log on/log off Hamlet while not filtering at all. - - Filter again and then click somewhere else. + - Use "q" hotkey to open the search. + - Filter for Hamlet, Prospero, Othello, etc. + - Log on Hamlet and log off Hamlet while filtering for Hamlet. + - Log on/log off Hamlet while filtering for Othello. + - Log on/log off Hamlet while not filtering at all. + - Filter again and then click somewhere else. -### Stream permissions ### +### Stream permissions This is an important category to test, because we obviously do not want to have bugs where people can read messages on streams they @@ -338,16 +352,16 @@ that Cordelia has the correct visibility to them. First, we start off with "positive" tests. - Positive tests - - Have Hamlet create a public stream w/Cordelia subscribed and - have him post a message to the stream. - - Have Hamlet create a public stream without Cordelia and then... - - Have Hamlet post to the stream. - - Have Cordelia subscribe to the stream. - - Verify Cordelia can see the previous message. - - Have Cordelia post a message to the stream. - - Have Hamlet create a private stream with Cordelia - invited and test a two-way conversation between the two - users. + - Have Hamlet create a public stream w/Cordelia subscribed and + have him post a message to the stream. + - Have Hamlet create a public stream without Cordelia and then... + - Have Hamlet post to the stream. + - Have Cordelia subscribe to the stream. + - Verify Cordelia can see the previous message. + - Have Cordelia post a message to the stream. + - Have Hamlet create a private stream with Cordelia + invited and test a two-way conversation between the two + users. For negative tests, we want to dig a little deeper to find back doors for Cordelia to access the stream. Here are some techniques @@ -364,21 +378,21 @@ and she can subsequently subscribe. For private streams, she should not even know they exist (until she's invited, of course). - Negative tests - - Have Hamlet create a public stream without inviting Cordelia. - - Verify Cordelia can see the stream in her settings. - - Verify Cordelia can't compose a message to the stream. - - Verify that Cordelia sees nothing when Hamlet posts to - the stream. - - Have Hamlet create a public stream with Cordelia, but then - have Iago revoke her subscription using the admin page. - - Verify that the stream appears in Cordelia's left sidebar - and then goes away. - - Try to have Cordelia view the stream using a sneaky - search along the lines of `stream:foo`. - - Have Hamlet create a private stream without inviting Cordelia. - - Verify Cordelia can't compose a message to the stream. + - Have Hamlet create a public stream without inviting Cordelia. + - Verify Cordelia can see the stream in her settings. + - Verify Cordelia can't compose a message to the stream. + - Verify that Cordelia sees nothing when Hamlet posts to + the stream. + - Have Hamlet create a public stream with Cordelia, but then + have Iago revoke her subscription using the admin page. + - Verify that the stream appears in Cordelia's left sidebar + and then goes away. + - Try to have Cordelia view the stream using a sneaky + search along the lines of `stream:foo`. + - Have Hamlet create a private stream without inviting Cordelia. + - Verify Cordelia can't compose a message to the stream. -### Search ### +### Search The main task for testing search is to play around with search suggestions (autocomplete). Once you select an option, verify the @@ -387,6 +401,7 @@ reflects the current narrow. If a search comes up legitimately empty, have Hamlet send a message that matches the search. Here are searches you should be able to do with autocomplete: + - stream:design - stream:Verona topic:Verona1 - stream:Verona keyword @@ -398,17 +413,20 @@ Here are searches you should be able to do with autocomplete: - PMs with Hamlet matching keyword "foo" There are some things you can try that don't come up in autocomplete: + - -stream:Verona (exclude Verona) - stream:Verona stream:devel (should return no results) Miscellaneous: + - Use the "/" hotkey to start a search. - Use the "x" icon to clear a search. - Use the "Esc" hotkey to clear a search. -### Stream settings ### +### Stream settings Test various UI entry points into stream settings: + - Use small gear menu in left sidebar, then filter to "devel". - Use popover menu in left sidebar next to "devel". - Use gear menu above buddy list and filter to "devel". @@ -417,6 +435,7 @@ Test various UI entry points into stream settings: (I'm not sure why we still have the chevron at this writing.) Create new public stream "public1" and add Hamlet: + - Type "public1" in the text box and then click "Create new stream." - Select "People must be invited" and then verify you can't select "Announce stream". @@ -425,6 +444,7 @@ Create new public stream "public1" and add Hamlet: - Hit the "Create" button. Test subscribe/unsubscribe: + - Log in as Hamlet and go to his stream settings. - As Cordelia, unsubscribe from "public1" using the checkmark in the streams settings page. @@ -435,16 +455,18 @@ Test subscribe/unsubscribe: As Cordelia, exercise different options in Create Stream dialog by creating streams s1, s2, s3, etc.: + - s1: anyone can join, announce it, and add Hamlet using filter feature - s2: people must be invited - s3: anyone can join, don't announce - s4: check all, then uncheck all, then invite only Hamlet - s5: invite everybody but Hamlet - s6: - - create the stream as public, but don't subscribe anybody initially - - then click on stream options to add Hamlet using "Add" button + - create the stream as public, but don't subscribe anybody initially + - then click on stream options to add Hamlet using "Add" button Test per-stream options: + - Use "devel" stream and send a message to it - Do mute and unmute, have Hamlet send messages - Test notifications on/off, have Hamlet send messages @@ -453,50 +475,51 @@ Test per-stream options: messages view - Verify stream subscriber counts in the main stream view -### User settings ### +### User settings You can modify per-user settings by choosing "Settings" in the gear menu. Do these tasks as Cordelia. - Your account - - Change full name (Hamlet should see the name change) - - Customize profile picture - - Deactivate account (and then log in as Iago to re-activate Cordelia) + - Change full name (Hamlet should see the name change) + - Customize profile picture + - Deactivate account (and then log in as Iago to re-activate Cordelia) - Display settings - - Right now, these unfortunately require reloads to take effect. - - Default language (change to Spanish) - - Show user list on left sidebar in narrow windows (verify by making window thinner) - - 24-hour time (and then test going back to AM/PM) + - Right now, these unfortunately require reloads to take effect. + - Default language (change to Spanish) + - Show user list on left sidebar in narrow windows (verify by making window thinner) + - 24-hour time (and then test going back to AM/PM) - Notifications - - Stream message - - turn off notifications at user level - - create a new stream - - have Hamlet send a message - - turn on notifications at user level - - create a new stream - - have Hamlet send a message - - then turn off notifications for that stream - - have Hamlet send another message - - Private messages and @-mentions - - Test Desktop/Audible options - - You can ignore other stuff for now + - Stream message + - turn off notifications at user level + - create a new stream + - have Hamlet send a message + - turn on notifications at user level + - create a new stream + - have Hamlet send a message + - then turn off notifications for that stream + - have Hamlet send another message + - Private messages and @-mentions + - Test Desktop/Audible options + - You can ignore other stuff for now - Bots/API key - - Create a bot with a generic avatar and send it a PM - - Create a bot with a custom avatar and send it a PM - - Change your API key + - Create a bot with a generic avatar and send it a PM + - Create a bot with a custom avatar and send it a PM + - Change your API key - Alert words - - Create an alert word - - Have Hamlet send you a message that includes the alert word + - Create an alert word + - Have Hamlet send you a message that includes the alert word - Zulip labs - - Turn on auto-scroll to new messages (and have Hamlet send you one) - - Turn on/off "Enable desktop notifications for new streams" and test. - (We may eliminate this option soon.) + - Turn on auto-scroll to new messages (and have Hamlet send you one) + - Turn on/off "Enable desktop notifications for new streams" and test. + (We may eliminate this option soon.) -### Keyboard shortcuts ### +### Keyboard shortcuts We mostly test keyboard shortcuts as part of other tasks. Here are the tasks for this section: + - Use the "?" hotkey to open the keyboard help - Proofread the dialog for typos. - Close the dialog. @@ -504,20 +527,22 @@ Here are the tasks for this section: - Find a hotkey that you don't frequently use and experiment with its usage. -### Miscellaneous menu options ### +### Miscellaneous menu options Make sure that these options launch appropriate help screens: -- Proofread and try a couple random options: - - Message formatting - - Search operators -- Make sure help launches in a separate browser tab: - - Desktop and mobile apps - - Integrations - - API documentation -### Inviting users/tutorial ### +- Proofread and try a couple random options: + - Message formatting + - Search operators +- Make sure help launches in a separate browser tab: + - Desktop and mobile apps + - Integrations + - API documentation + +### Inviting users/tutorial Here are the tasks: + - Invite ignore@zulip.com using the link beneath the buddy list but then don't take further action. - Fully invite foo@zulip.com using the gear menu. @@ -529,10 +554,11 @@ Here are the tasks: much of our production code, since the login flow is customized for the development environment). -### To be continued... ### +### To be continued... This document does not cover settings/admin options yet. The main things to do when testing the settings system are: + - Verify that changes are synced to other users. - Verify error messages appear if you do something wrong and look right. - For organization settings, verify that they look right in read-only diff --git a/docs/testing/philosophy.md b/docs/testing/philosophy.md index 064e8bda64..a44a1e9fbf 100644 --- a/docs/testing/philosophy.md +++ b/docs/testing/philosophy.md @@ -61,18 +61,19 @@ suite (`test-js-with-node`) to run in under 10 seconds. It'd be a long blog post to summarize everything we do to help achieve these goals, but a few techniques are worth highlighting: + - Our test suites are designed to not access the Internet, since the Internet might be down or unreliable in the test environment. Where outgoing HTTP requests are required to test something, we mock the responses with libraries like `responses`. - We carefully avoid the potential for contamination of data inside services like PostgreSQL, Redis, and memcached from different tests. - - Every test case prepends a unique random prefix to all keys it - uses when accessing Redis and memcached. - - Every test case runs inside a database transaction, which is - aborted after the test completes. Each test process interacts - only with a fresh copy of a special template database used for - server tests that is destroyed after the process completes. + - Every test case prepends a unique random prefix to all keys it + uses when accessing Redis and memcached. + - Every test case runs inside a database transaction, which is + aborted after the test completes. Each test process interacts + only with a fresh copy of a special template database used for + server tests that is destroyed after the process completes. - We rigorously investigate non-deterministically failing tests as though they were priority bugs in the product. @@ -141,6 +142,7 @@ Some examples of this philosophy: run, and produce this output". In the Zulip context: + - Zulip uses the same API for our webapp as for our mobile clients and third-party API clients, and most of our server tests are written against the Zulip API. @@ -150,6 +152,7 @@ In the Zulip context: message as output, to test the actual interface. So, to summarize our approach to integration vs. unit testing: + - While we aim to achieve test coverage of every significant code path in the Zulip server, which is commonly associated with unit testing, most of our tests are integration tests in the sense of sending a diff --git a/docs/testing/testing-with-django.md b/docs/testing/testing-with-django.md index 25644fd831..ed5afae3ca 100644 --- a/docs/testing/testing-with-django.md +++ b/docs/testing/testing-with-django.md @@ -120,11 +120,12 @@ Here are some example action methods that tests may use for data setup: Some tests need to access the filesystem (e.g. `test_upload.py` tests for `LocalUploadBackend` and the data import tests). Doing this correctly requires care to avoid problems like: + - Leaking files after every test (which are clutter and can eventually -run the development environment out of disk) or + run the development environment out of disk) or - Interacting with other parallel processes of this `test-backend` run -(or another `test-backend` run), or with later tests run by this -process. + (or another `test-backend` run), or with later tests run by this + process. To avoid these problems, you can do the following: @@ -151,8 +152,8 @@ techniques. #### What is mocking? -When writing tests, *mocks allow you to replace methods or objects with fake entities -suiting your testing requirements*. Once an object is mocked, **its original code does not +When writing tests, _mocks allow you to replace methods or objects with fake entities +suiting your testing requirements_. Once an object is mocked, **its original code does not get executed anymore**. Rather, you can think of a mocked object as an initially empty shell: @@ -201,12 +202,12 @@ def greet(name_key: str) -> str: ``` -> **You have a problem**: `greet()` calls `fetch_database()`. `fetch_database()` does some look-ups in - a database. *You haven't created that database for your tests, so your test would fail, even though - the code is correct.* +a database. _You haven't created that database for your tests, so your test would fail, even though +the code is correct._ - Luckily, you know that `fetch_database("Mario")` should return "Mr. Mario Mario". - - *Hint*: Sometimes, you might not know the exact return value, but one that is equally valid and works + - _Hint_: Sometimes, you might not know the exact return value, but one that is equally valid and works with the rest of the code. In that case, just use this one. -> **Solution**: You mock `fetch_database()`. This is also referred to as "mocking out" `fetch_database()`. @@ -233,7 +234,7 @@ It also implements `MagicMock`, which is the same as `Mock`, but contains many d those are the ones starting with with a dunder `__`). From the docs: > In most of these examples the Mock and MagicMock classes are interchangeable. As the MagicMock is the more capable class - it makes a sensible one to use by default. +> it makes a sensible one to use by default. `Mock` itself is a class that principally accepts and records any and all calls. A piece of code like @@ -246,7 +247,7 @@ foo.baz foo.qux = 42 ``` -is *not* going to throw any errors. Our mock silently accepts all these calls and records them. +is _not_ going to throw any errors. Our mock silently accepts all these calls and records them. `Mock` also implements methods for us to access and assert its records, e.g. ```python @@ -255,8 +256,8 @@ foo.bar.assert_called_with('quux') Finally, `unittest.mock` also provides a method to mock objects only within a scope: `patch()`. We can use `patch()` either as a decorator or as a context manager. In both cases, the mock created by `patch()` will apply for the scope of the decorator / -context manager. `patch()` takes only one required argument `target`. `target` is a string in dot notation that *refers to -the name of the object you want to mock*. It will then assign a `MagicMock()` to that object. +context manager. `patch()` takes only one required argument `target`. `target` is a string in dot notation that _refers to +the name of the object you want to mock_. It will then assign a `MagicMock()` to that object. As an example, look at the following code: ```python @@ -269,7 +270,7 @@ with mock.patch('__main__.urandom', return_value=42): print(urandom(1)) # We exited the context manager, so the mock doesn't apply anymore. Will return a random byte. ``` -*Note that calling `mock.patch('os.urandom', return_value=42)` wouldn't work here*: `os.urandom` would be the name of our patched +_Note that calling `mock.patch('os.urandom', return_value=42)` wouldn't work here_: `os.urandom` would be the name of our patched object. However, we imported `urandom` with `from os import urandom`; hence, we bound the `urandom` name to our current module `__main__`. @@ -454,28 +455,28 @@ to verify that the endpoint is properly failing. Here are some things to consider when writing new tests: - **Duplication** We try to avoid excessive duplication in tests. -If you have several tests repeating the same type of test setup, -consider making a setUp() method or a test helper. + If you have several tests repeating the same type of test setup, + consider making a setUp() method or a test helper. - **Network independence** Our tests should still work if you don't -have an internet connection. For third party clients, you can simulate -their behavior using fixture data. For third party servers, you can -typically simulate their behavior using mocks. + have an internet connection. For third party clients, you can simulate + their behavior using fixture data. For third party servers, you can + typically simulate their behavior using mocks. - **Coverage** We have 100% line coverage on several of our backend -modules. You can use the `--coverage` option to generate coverage -reports, and new code should have 100% coverage, which generally -requires testing not only the "happy path" but also error handling -code and edge cases. It will generate a nice HTML report that you can -view right from your browser (the tool prints the URL where the report -is exposed in your development environment). + modules. You can use the `--coverage` option to generate coverage + reports, and new code should have 100% coverage, which generally + requires testing not only the "happy path" but also error handling + code and edge cases. It will generate a nice HTML report that you can + view right from your browser (the tool prints the URL where the report + is exposed in your development environment). - **Console output** A properly written test should print nothing to -the console; use `with self.assertLogs` to capture and verify any -logging output. Note that we reconfigure various loggers in -`zproject/test_extra_settings.py` where the output is unlikely to be -interesting when running our test suite. -`test-backend --ban-console-output` checks for stray print statements. + the console; use `with self.assertLogs` to capture and verify any + logging output. Note that we reconfigure various loggers in + `zproject/test_extra_settings.py` where the output is unlikely to be + interesting when running our test suite. + `test-backend --ban-console-output` checks for stray print statements. Note that `test-backend --coverage` will assert that various specific files in the project have 100% test coverage and diff --git a/docs/testing/testing-with-node.md b/docs/testing/testing-with-node.md index da8097a080..25678a0187 100644 --- a/docs/testing/testing-with-node.md +++ b/docs/testing/testing-with-node.md @@ -7,6 +7,7 @@ system since it is much (>100x) faster and also easier to do correctly than the Puppeteer system. You can run this test suite as follows: + ```bash tools/test-js-with-node ``` @@ -101,10 +102,10 @@ code you're trying to test. For that reason, each unit test file explicitly declares all of the modules it depends on, with a few different types of declarations depending on whether we want to: -- Exercise the module's real code for deeper, more realistic testing? -- Stub out the module's interface for more control, speed, and - isolation? -- Do some combination of the above? +- Exercise the module's real code for deeper, more realistic testing? +- Stub out the module's interface for more control, speed, and + isolation? +- Do some combination of the above? For all the modules where you want to run actual code, add statements like the following toward the top of your test file: @@ -164,7 +165,7 @@ tools/test-js-with-node --coverage If tests pass, you will get instructions to view coverage reports in your browser. -Note that modules that we don't test *at all* aren't listed in the +Note that modules that we don't test _at all_ aren't listed in the report, so this tends to overstate how good our overall coverage is, but it's accurate for individual files. You can also click a filename to see the specific statements and branches not tested. 100% branch @@ -191,31 +192,32 @@ These instructions assume you're using the Vagrant development environment. 2. In WebStorm, navigate to `Preferences -> Tools -> Vagrant` and configure the following: - - `Instance folder` should be the root of the `zulip` repository on - your host (where the Vagrantfile is located). - - `Provider` should be `virtualbox` on macOS and Docker on Linux - - In `Boxes`, choose the one used for Zulip (unless you use - Virtualbox for other things, there should only be one option). + - `Instance folder` should be the root of the `zulip` repository on + your host (where the Vagrantfile is located). + - `Provider` should be `virtualbox` on macOS and Docker on Linux + - In `Boxes`, choose the one used for Zulip (unless you use + Virtualbox for other things, there should only be one option). - You shouldn't need to set these additional settings: - - `Vagrant executable` should already be correctly `vagrant`. - - `Environment Variables` is not needed. + You shouldn't need to set these additional settings: + + - `Vagrant executable` should already be correctly `vagrant`. + - `Environment Variables` is not needed. 3. You'll now need to set up a WebStorm "Debug Configuration". Open the `Run/Debug Configuration` menu and create a new `Node.js` config: - 1. Under `Node interpreter:` click the 3 dots to the right side and + 1. Under `Node interpreter:` click the 3 dots to the right side and click on the little plus in the bottom left of the `Node.js Interpreters` window. - 1. Select `Add Remote...`. - 1. In the `Configure Node.js Remote Interpreter`, window select `Vagrant` - 1. Wait for WebStorm to connect to Vagrant. This will be displayed - by the `Vagrant Host URL` section updating to contain the Vagrant - SSH URL, e.g. `ssh://vagrant@127.0.0.1:2222`. - 1. **Set the `Node.js interpreter path` to `/usr/local/bin/node`** - 1. Hit `OK` 2 times to get back to the `Run/Debug Configurations` window. - 1. Under `Working Directory` select the root `zulip` directory. - 1. Under `JavaScript file`, enter `frontend_tests/zjsunit/index.js` - -- this is the root script for Zulip's node unit tests. + 1. Select `Add Remote...`. + 1. In the `Configure Node.js Remote Interpreter`, window select `Vagrant` + 1. Wait for WebStorm to connect to Vagrant. This will be displayed + by the `Vagrant Host URL` section updating to contain the Vagrant + SSH URL, e.g. `ssh://vagrant@127.0.0.1:2222`. + 1. **Set the `Node.js interpreter path` to `/usr/local/bin/node`** + 1. Hit `OK` 2 times to get back to the `Run/Debug Configurations` window. + 1. Under `Working Directory` select the root `zulip` directory. + 1. Under `JavaScript file`, enter `frontend_tests/zjsunit/index.js` + -- this is the root script for Zulip's node unit tests. Congratulations! You've now set up the integration. diff --git a/docs/testing/testing-with-puppeteer.md b/docs/testing/testing-with-puppeteer.md index 97d0af5cb7..9fcab8c94b 100644 --- a/docs/testing/testing-with-puppeteer.md +++ b/docs/testing/testing-with-puppeteer.md @@ -10,6 +10,7 @@ keyboard shortcuts, etc.). ## Running tests You can run this test suite as follows: + ```bash tools/test-js-with-puppeteer ``` diff --git a/docs/testing/testing.md b/docs/testing/testing.md index 248ec16336..cb9b1b5631 100644 --- a/docs/testing/testing.md +++ b/docs/testing/testing.md @@ -140,12 +140,12 @@ depending on Internet access. This enforcement code results in the following exception: - ```pytb - File "tools/test-backend", line 120, in internet_guard - raise Exception("Outgoing network requests are not allowed in the Zulip tests." - Exception: Outgoing network requests are not allowed in the Zulip tests. - ... - ``` +```pytb +File "tools/test-backend", line 120, in internet_guard + raise Exception("Outgoing network requests are not allowed in the Zulip tests." +Exception: Outgoing network requests are not allowed in the Zulip tests. +... +``` #### Documentation tests diff --git a/docs/testing/typescript.md b/docs/testing/typescript.md index 1a18190bcb..2215e0935f 100644 --- a/docs/testing/typescript.md +++ b/docs/testing/typescript.md @@ -15,7 +15,7 @@ discussion and very much subject to change. A typical piece of TypeScript code looks like this: -``` ts +```ts setdefault(key: K, value: V): V { const mapping = this._items[this._munge(key)]; if (mapping === undefined) { @@ -29,7 +29,6 @@ The following resources are valuable for learning TypeScript: - The main documentation on [TypeScript syntax][typescript-handbook]. - ## Type checking TypeScript types are checked by the TypeScript compiler, `tsc`, which diff --git a/docs/translating/chinese.md b/docs/translating/chinese.md index 3b290a819f..cf67293997 100644 --- a/docs/translating/chinese.md +++ b/docs/translating/chinese.md @@ -8,9 +8,9 @@ Zulip is a modern internet application, many Chinese translations are borrowed from the popular Web software, such as WeiBo, WeChat, QQ Mail etc. that most Chinese users are familiar with. -Zulip的文风比较口语化，考虑到大多数中国用户的习惯，翻译时的语言习惯稍 -微正式了一点，但也尽量避免刻板。Zulip是一款时尚的互联网应用，翻译时也 -借鉴了中国用户熟悉的微博、微信、QQ邮箱等软件的用语习惯，以期贴近用户。 +Zulip 的文风比较口语化，考虑到大多数中国用户的习惯，翻译时的语言习惯稍 +微正式了一点，但也尽量避免刻板。Zulip 是一款时尚的互联网应用，翻译时也 +借鉴了中国用户熟悉的微博、微信、QQ 邮箱等软件的用语习惯，以期贴近用户。 ## Terms(术语) @@ -23,9 +23,9 @@ translated as "私信". The domestic WeiBo, WeChat also keep in line with the habit. "Starred Message" is similar to "Star Mail (星标邮件)" feature in QQ Mail, so it is translated into "星标消息". -Message可直译为“消息”、“信息”等，两者皆可，这里统一选用“消息”。例如， +Message 可直译为“消息”、“信息”等，两者皆可，这里统一选用“消息”。例如， “Stream Message”译作“频道消息”；但“Private Message”又译为“私信"，与国 -内微博、微信的使用习惯保持一致。“Starred Message”类似于QQ邮箱中的“星标 +内微博、微信的使用习惯保持一致。“Starred Message”类似于 QQ 邮箱中的“星标 邮件”功能，这里也借鉴翻译为“星标消息”。 - Stream - **频道** @@ -41,10 +41,10 @@ group. However, "讨论组" has one more Chinese character than "频道 (Channel)". 曾经使用的翻译有“群组”、“主题”、“版块”，还有“栏目”。现在选择的“频道”灵 -感来源于Ingress游戏中的聊天“Channel”。因为“Stream”可以“新建/删除 +感来源于 Ingress 游戏中的聊天“Channel”。因为“Stream”可以“新建/删除 （Create/Delete）”、也可以“订阅/退订（Subscribe/Unsubscribe）”， “Stream”内部还可以发起“话题（Topic）讨论。“Stream”还有一个备选方案，就 -是“讨论组”，字多一个，稍微有点啰嗦。主要参考自以前QQ的“讨论组”，在QQ中 +是“讨论组”，字多一个，稍微有点啰嗦。主要参考自以前 QQ 的“讨论组”，在 QQ 中 是一种临时的群组。 - Topic - **话题** @@ -69,7 +69,7 @@ integrating Zulip production with other applications and services. For integrity in Chinese expression, it is translated as "应用整合 (Application Integration)". -“Integration”原意为“集成”与“整合”，这里表示将其它的应用或者服务与Zulip +“Integration”原意为“集成”与“整合”，这里表示将其它的应用或者服务与 Zulip 实现整合。为表达意思完整，补充翻译为“应用整合”。 - Notification - **通知** @@ -108,8 +108,8 @@ readability considerations. 1. 在汉语中，“筛选”表示按照指定条件进行挑选的方式。“Narrow to ...”的含 义为“使...缩小范围”，两者有一定共通性。 -2. “筛选”也是比较大众化的计算机用语，易于为大家所接受。例如Microsoft - Excel中的“筛选”功能。 +2. “筛选”也是比较大众化的计算机用语，易于为大家所接受。例如 Microsoft + Excel 中的“筛选”功能。 另外，在搜索功能的语境中，“Narrow to ...”没有翻译为“筛选”，而翻译为“搜 索”，这是出于可读性的考虑。 @@ -121,7 +121,7 @@ set. Such a translation is not appropriate for Zulip. "开启/关闭免打 扰(Turn off/on Notification)" is a sense to sense translation, which is also borrowed from the WeChat. -“Mute”常见的中文翻译为“静音”，在电视设备中常见，用在Zulip中并不太合适。 +“Mute”常见的中文翻译为“静音”，在电视设备中常见，用在 Zulip 中并不太合适。 这里取意译，与大家常用的微信（WeChat）中“消息免打扰”用语习惯一致。 - Deactivate/Reactivate - **禁用/启用(帐户)，关闭/激活(社区)** @@ -141,11 +141,11 @@ example "Your realm has been deactivated." to "您的社区已关闭". for general users. Other translations "错误(Error)", "非法(Illegal)", "不合法(Invalid)" are all ok. Generally, it is translated as "不正确 (Incorrect)" for consistency. For example, "Invalid API key" is -translated as "API码不正确". +translated as "API 码不正确". “Invalid”大部分用于一些异常信息，这些信息普通用户应该很少见到。可选翻 译有“错误”、“非法”、“不合法”；为保持一致的习惯，这里统一翻译为“不正确”。 -例如“Invalid API key”翻译为“API码不正确”。 +例如“Invalid API key”翻译为“API 码不正确”。 - I want - **开启** @@ -196,6 +196,6 @@ the dot in Chinese (。) often has a bad effect on page layout. It is recommended to omit the dot, just leave empty at the end of the sentence or paragraph. -感叹号在Zulip中出现非常多，可能英文中感叹号的语气比中文中略轻一点。在 +感叹号在 Zulip 中出现非常多，可能英文中感叹号的语气比中文中略轻一点。在 中文翻译建议省略大部分的感叹号。另外，句号在中文排版中比较影响美观，因 此也一般建议省略不翻。句末留空即可。 diff --git a/docs/translating/french.md b/docs/translating/french.md index cad5df0b74..6ad4bfb60c 100644 --- a/docs/translating/french.md +++ b/docs/translating/french.md @@ -2,10 +2,10 @@ ## Rules -- Use of *vous* instead of *tu*, +- Use of _vous_ instead of _tu_, - A space before and after a colon and a semi-colon and a space after a dot and a comma, - Follow english capitalization, -- Prefer the infinitive form of a verb: *Save* into *Sauver* (instead of *Sauvez*). +- Prefer the infinitive form of a verb: _Save_ into _Sauver_ (instead of _Sauvez_). Some translations can be tricky, so please don't hesitate to ask the community or contribute to this guide. diff --git a/docs/translating/german.md b/docs/translating/german.md index b746e48f5b..a95611ab1f 100644 --- a/docs/translating/german.md +++ b/docs/translating/german.md @@ -4,7 +4,6 @@ Thank you for considering to contribute to the German translation! Before you start writing, please make sure that you have read the following general translation rules. - ## Rules ### Formal or informal? @@ -17,27 +16,27 @@ in a more colloquial style, German translations should be rather informal as wel **Don't use slang or regional phrases in the German translation:** -- Instead of *"So'n Dreck kann jedem mal passieren."*, you could -say *"Dieser Fehler tritt häufiger auf."* +- Instead of _"So'n Dreck kann jedem mal passieren."_, you could + say _"Dieser Fehler tritt häufiger auf."_ -- "Das ist die Seite, wo der Quelltext steht." - the "*wo*" is regional, -say *"Das ist die Seite, auf der der Quelltext steht."* instead. +- "Das ist die Seite, wo der Quelltext steht." - the "_wo_" is regional, + say _"Das ist die Seite, auf der der Quelltext steht."_ instead. ### Form of address **Use "Du" instead of "Sie".** For the reasons provided in [the previous section](#formal-or-informal), -stick to *Du* (informal) instead of *Sie* (formal) when addressing -the reader and remember to capitalize *Du*. +stick to _Du_ (informal) instead of _Sie_ (formal) when addressing +the reader and remember to capitalize _Du_. ### Form of instruction **Prefer imperative over constructions with auxiliary verbs.** -For instructions, try to use the imperative (e.g. *"Gehe auf die Seite"* - -*"Go to the page"*) instead of constructions with auxiliary verbs -(e.g. *"Du musst auf die Seite ... gehen"* - *"You have to go the page ..."*). +For instructions, try to use the imperative (e.g. _"Gehe auf die Seite"_ - +_"Go to the page"_) instead of constructions with auxiliary verbs +(e.g. _"Du musst auf die Seite ... gehen"_ - _"You have to go the page ..."_). This keeps the phrases short, less stiff and avoids unnecessary addressing of the reader. @@ -47,25 +46,25 @@ of the reader. To be consistent with other online platforms, use continuous labels for buttons, item titles, etc. with verbs in infinitive form, -e.g. *Manage streams* - *Kanäle verwalten* instead of *Verwalte Kanäle*. +e.g. _Manage streams_ - _Kanäle verwalten_ instead of _Verwalte Kanäle_. ### Concatenation of words **Try to avoid it.** German is famous for its concatenations of nouns -(e.g. *Heizölrückstoßdämpfung*, which means *fuel oil recoil attenuation*). +(e.g. _Heizölrückstoßdämpfung_, which means _fuel oil recoil attenuation_). For the sake of correct rendering and simplicity, you should try to avoid such concatenations whenever possible, since they can break the layout of the Zulip frontend. Try to stick to a maximum length of 20 characters and follow your intuition. -- A term like *Tastaturkürzel* for *Keyboard shortcuts* is fine - it is -shorter than 20 characters and commonly used in web applications. +- A term like _Tastaturkürzel_ for _Keyboard shortcuts_ is fine - it is + shorter than 20 characters and commonly used in web applications. -- A term like *Benachrichtigungsstichwörter* for *Alert words* should -not be used, it sounds odd and is longer than 20 characters. -You could use "*Stichwörter, die mich benachrichtigen*" instead. +- A term like _Benachrichtigungsstichwörter_ for _Alert words_ should + not be used, it sounds odd and is longer than 20 characters. + You could use "_Stichwörter, die mich benachrichtigen_" instead. ### Anglicisms @@ -76,21 +75,21 @@ This becomes even more evident in internet applications, so you should not be afraid of using them if they provide an advantage over the German equivalent. Take the following two examples as a reference: -- Translating *Stream*: Use the German word *Kanal*, since it is just as short -and used in other web apps. +- Translating _Stream_: Use the German word _Kanal_, since it is just as short + and used in other web apps. -- Translating *Bot*: Use *Bot*, as a completely accurate German -equivalent **doesn't** exist (e.g. *Roboter*) and the term *Bot* is not -unknown to German speakers. +- Translating _Bot_: Use _Bot_, as a completely accurate German + equivalent **doesn't** exist (e.g. _Roboter_) and the term _Bot_ is not + unknown to German speakers. ### Special characters **Use "ä, ö, ü" and "ß" consistently.** -While *ä, ö, ü* and *ß* are more and more being replaced by *ae, oe, ue* -and *ss* in chats, forums and even websites, German translations +While _ä, ö, ü_ and _ß_ are more and more being replaced by _ae, oe, ue_ +and _ss_ in chats, forums and even websites, German translations containing umlauts have a more trustworthy appearance. -For capitalizations, you can replace the *ß* by *ss*. +For capitalizations, you can replace the _ß_ by _ss_. ### False friends @@ -100,36 +99,35 @@ A false friend is a word in another language that is spelled or sounds similar to a word in one's own language, yet has a different meaning. False friends for the translation from German to English include -*actually* - *eigentlich*, *eventually* - *schließlich*, *map* - *Karte*, etc. +_actually_ - _eigentlich_, _eventually_ - _schließlich_, _map_ - _Karte_, etc. Make sure to not walk into such a trap. ### Other - Try to keep words and phrases short and understandable. The front-end -developers will thank you ;) + developers will thank you ;) - Be consistent. Use the same terms for the same things, even if that -means repeating. Have a look at other German translations on Zulip -to get a feeling for the vocabulary. + means repeating. Have a look at other German translations on Zulip + to get a feeling for the vocabulary. - Balance common verbs and nouns with specific IT-related translations -of English terms - this can be tricky, try to check how other resources -were translated (e.g. Gmail, Microsoft websites, Facebook) to decide -what wouldn't sound awkward / rude in German. + of English terms - this can be tricky, try to check how other resources + were translated (e.g. Gmail, Microsoft websites, Facebook) to decide + what wouldn't sound awkward / rude in German. - For additional translation information, feel free to check out -[this](https://en.wikipedia.org/wiki/Wikipedia:Translating_German_WP) Wikipedia guide -on translating German Wikipedia articles into English. + [this](https://en.wikipedia.org/wiki/Wikipedia:Translating_German_WP) Wikipedia guide + on translating German Wikipedia articles into English. Some terms are very tricky to translate, so be sure to communicate with other German speakers in the community. It's all about making Zulip friendly and usable. - ## Terms (Begriffe) - Message - **Nachricht** -*"Nachricht" (Facebook, WhatsApp, Transifex)* +_"Nachricht" (Facebook, WhatsApp, Transifex)_ - Private Message (PM) - **Private Nachricht (PN)** @@ -137,17 +135,17 @@ Since we try to avoid concatenating words whenever possible, don't use "Privatnachricht" . PN is the officially used abbreviation for "Private Nachricht" and is used in many German chat forums. -*"Private Nachricht" (YouTube, Transifex)* +_"Private Nachricht" (YouTube, Transifex)_ - Starred Message - **Markierte Nachricht** We go with "markiert" instead of "gesternt" (which is not even a proper German word) here, since it comes closer to the original meaning of "starred". -*"Markierte Nachricht" (Gmail, Transifex), -"Nachricht mit Stern" (WhatsApp)* +_"Markierte Nachricht" (Gmail, Transifex), +"Nachricht mit Stern" (WhatsApp)_ -*"Bereich" (Transifex), "Community" (Google+)* +_"Bereich" (Transifex), "Community" (Google+)_ - Stream - **Stream** @@ -158,24 +156,23 @@ skills, and the best choice for describing Zulip's chat hierarchy. The term by other chat applications with a simple, flat chat hierarchy, that is, no differentiation between streams and topics. -*"Stream" (Transifex), "Kanal" (KDE IRC documentation, various -small German forums)* +_"Stream" (Transifex), "Kanal" (KDE IRC documentation, various +small German forums)_ - Topic - **Thema** -*(Gmail - for email subjects, Transifex)* +_(Gmail - for email subjects, Transifex)_ - Invite-Only Stream - **Geschlossener Stream** For users to be able to join to an "invite-only" stream, they must have been invited by some user in this stream. This type of stream is equivalent to Facebook's "closed" groups, which in turn translates to "geschlossen" in German. -This translation seems to be appropriate, for example [Linguee]( -https://www.linguee.de/englisch-deutsch/uebersetzung/invite-only.html) +This translation seems to be appropriate, for example [Linguee](https://www.linguee.de/englisch-deutsch/uebersetzung/invite-only.html) search returns only paraphrases of this term. -*"Geschlossener Stream" (Transifex), "Geschlossene Gruppe" (Facebook), -paraphrases (Linguee)* +_"Geschlossener Stream" (Transifex), "Geschlossene Gruppe" (Facebook), +paraphrases (Linguee)_ - Public Stream - **Öffentlicher Stream** @@ -183,14 +180,14 @@ While some might find this direct translation a tad long, the alternative "Offener Stream" can be ambiguous - especially users who are inexperienced with Zulip could think of this as streams that are online. -*"Öffentlicher Stream" (Transifex)* +_"Öffentlicher Stream" (Transifex)_ - Bot - **Bot** Not only is "bot" a short and easily memorable term, it is also widely used in German technology magazines, forums, etc. -*"Bot" (Transifex, Heise, Die Zeit)* +_"Bot" (Transifex, Heise, Die Zeit)_ - Integration - **Integration** @@ -200,34 +197,34 @@ use "Integrationen" instead of "Integrations" when speaking of multiple integrations in German. There aren't many German sources available for this translation, but "Integration" has the same meaning in German and English. -*"Integration/-en" (Transifex)* +_"Integration/-en" (Transifex)_ - Notification - **Benachrichtigung** Nice and easy. Other translations for "notification" like "Erwähnung", "Bescheid" or "Notiz" don't fit here. -*"Benachrichtigung" (Facebook, Gmail, Transifex, Wikipedia)* +_"Benachrichtigung" (Facebook, Gmail, Transifex, Wikipedia)_ - Alert Word - **Signalwort** This one is tricky, since one might initially think of "Alarmwort" as a proper translation. "Alarm", however, has a negative connotation, people link it to unpleasant events. "Signal", on the other hand, is neutral, just like -"alert word". Nevertheless, [Linguee]( -https://www.linguee.de/deutsch-englisch/search?source=auto&query=alert+word) +"alert word". Nevertheless, [Linguee](https://www.linguee.de/deutsch-englisch/search?source=auto&query=alert+word) shows that some websites misuse "Alarm" for the translation. -*"Signalwort" (Transifex), "Wort-Alarm" (Linguee)* +_"Signalwort" (Transifex), "Wort-Alarm" (Linguee)_ - View - **View** (Developer documentation) Since this is a Zulip-specific term for + > every path that the Zulip server supports (doesn’t show a 404 page for), and there is no German equivalent, talking of "Views" is preferable in the developer documentation and makes it easier to rely on parts of the German -*and* parts of the English documentation. +_and_ parts of the English documentation. - View - **Ansicht** (User documentation) @@ -237,13 +234,13 @@ For the user documentation, we want to use "Ansicht" instead of "view", as does not emphasize the developing aspects of views (in contrast to anglicisms, which Germans often link to IT-related definitions). -*"Ansicht" (Transifex)* +_"Ansicht" (Transifex)_ - Home - **Startseite** Nice and easy. "Zuhause" obviously doesn't fit here ;). -*"Startseite" (Facebook, Transifex)* +_"Startseite" (Facebook, Transifex)_ - Emoji - **Emoji** @@ -251,8 +248,7 @@ Nice and easy. "Zuhause" obviously doesn't fit here ;). "Bildschriftzeichen" (which exists!) would sound stiff and outdated. "Emoticon" works as well, but is not that common in German. -*"Emoji" (Facebook, WhatsApp), "Emoticon" (Google+)* - +_"Emoji" (Facebook, WhatsApp), "Emoticon" (Google+)_ ## Phrases (Ausdrücke) @@ -260,7 +256,7 @@ works as well, but is not that common in German. This translation is unambiguous. -*"Deabonnieren" (YouTube, Transifex)* +_"Deabonnieren" (YouTube, Transifex)_ - Narrow to - **Begrenzen auf** @@ -272,14 +268,14 @@ would be too long for many labels, the infinitive "begrenzen auf" is preferable. "einschränken auf" sounds equally good, but Transifex shows more use cases for "begrenzen auf". -*"Schränke auf ... ein." (Transifex) "Begrenze auf ... ." (Transifex)* +_"Schränke auf ... ein." (Transifex) "Begrenze auf ... ." (Transifex)_ - Filter - **Filtern** A direct translation is fine here. Watch out to to use the infinitive instead of the imperative, e.g. "Nachrichten filtern" instead of "Filtere Nachrichten". -*"Filtern" (Thunderbird, LinkedIn)* +_"Filtern" (Thunderbird, LinkedIn)_ - Mute/Unmute - **Stummschalten/Lautschalten** @@ -289,11 +285,11 @@ preferable due to its brevity. - Deactivate/Reactivate - **Deaktivieren/Reaktivieren** -*"Deaktivieren/Reaktivieren" (Transifex)* +_"Deaktivieren/Reaktivieren" (Transifex)_ - Search - **Suchen** -*"Suchen" (YouTube, Google, Facebook, Transifex)* +_"Suchen" (YouTube, Google, Facebook, Transifex)_ - Pin/Unpin - **Anpinnen/Loslösen** @@ -301,17 +297,17 @@ While "pinnen" is shorter than "anpinnen", "anpinnen" sweeps any amiguity out of the way. This term is not used too often on Zulip, so the length shouldn't be a problem. -*"Anpinnen/Ablösen" (Transifex), "Pinnen" (Pinterest)* +_"Anpinnen/Ablösen" (Transifex), "Pinnen" (Pinterest)_ - Mention/@mention - **Erwähnen/"@-Erwähnen** Make sure to say "@-erwähnen", but "die @-Erwähnung" (capitalized). -*"Erwähnen/@-Erwähnen" (Transifex)* +_"Erwähnen/@-Erwähnen" (Transifex)_ - Invalid - **Ungültig** -*"Ungültig" (Transifex)* +_"Ungültig" (Transifex)_ - Customization - **Anpassen** @@ -322,21 +318,21 @@ the infinitive form "anpassen". "Ich möchte" is the polite form of "Ich will". -*"Ich möchte" - (Transifex, general sense of politeness)* +_"Ich möchte" - (Transifex, general sense of politeness)_ - User - **Nutzer** "Benutzer" would work as well, but "Nutzer" is shorter and more commonly used in web applications. -*"Nutzer" (Facebook, Gmail), "Benutzer" (Transifex)* +_"Nutzer" (Facebook, Gmail), "Benutzer" (Transifex)_ - Person/People - Nutzer/Personen We use "Personen" instead of plural "Nutzer" for "people", as "Nutzer" stays the same in plural. -*"Nutzer/Personen" (Transifex)* +_"Nutzer/Personen" (Transifex)_ ## Other (Verschiedenes) @@ -347,11 +343,11 @@ informal language. If you would like to read more about the reasoning behind this, refer to the [general notes](#formal-or-informal) for translating German. -*"Du" (Google, Facebook), "Sie" (Transifex)* +_"Du" (Google, Facebook), "Sie" (Transifex)_ - We - **Wir** (rarely used) German guides don't use "wir" very often - they tend to reformulate the phrases instead. -*"Wir" (Google, Transifex)* +_"Wir" (Google, Transifex)_ diff --git a/docs/translating/hindi.md b/docs/translating/hindi.md index c2fe36e9e5..8b7b761c7d 100644 --- a/docs/translating/hindi.md +++ b/docs/translating/hindi.md @@ -2,13 +2,13 @@ Use informal Hindi for translation: -- Informal "you" (*तु*) instead of formal form *आप*. Many top software +- Informal "you" (_तु_) instead of formal form _आप_. Many top software companies (e.g. Google) use the informal one, because it's much more common in the daily language and avoids making translations look like they were written by machines. -- Imperative, active, and continuous verbs, e.g. *manage streams* - - *चैनल प्रबंधित करें*, not *चैनल प्रबंधन*. +- Imperative, active, and continuous verbs, e.g. _manage streams_ - + _चैनल प्रबंधित करें_, not _चैनल प्रबंधन_. - Warm and friendly phrasing whenever appropriate. @@ -24,6 +24,7 @@ with other Hindi speakers in the community. It's all about making Zulip friendly and usable. ## Terms(शर्तें) + - Message - **संदेश** - Private message (PM) - **निजी संदेश** - Stream - **धारा**: the use of the literal Hindi word for stream @@ -35,8 +36,8 @@ Zulip friendly and usable. - Bot - **बॉट** - Integration - **एकीकरण** - Notification - **अधिसूचना** -- Alert word - **सतर्क शब्द**: this is only *alert*. Nonetheless, adding *word* may - make the term confusing (something like *danger!* could be a "चेतावनी के शब्द" as well). +- Alert word - **सतर्क शब्द**: this is only _alert_. Nonetheless, adding _word_ may + make the term confusing (something like _danger!_ could be a "चेतावनी के शब्द" as well). Google Alerts uses "सतर्क शब्द" in its Hindi translation. - View - **राय** - Filter - **छानना**: as used with narrowing (see below). @@ -44,9 +45,10 @@ Zulip friendly and usable. - Emoji - **इमोजी** ## Phrases (वाक्यांशों) + - Subscribe/Unsubscribe to a stream - **एक धारा में सदस्यता लें/सदस्यता समाप्त करें** -- Narrow to - **अकेले फ़िल्टर करें**: this is *filter only*, because there's no other - word that's common enough in Hindi for *to narrow*. +- Narrow to - **अकेले फ़िल्टर करें**: this is _filter only_, because there's no other + word that's common enough in Hindi for _to narrow_. - Mute/Unmute - **शांत/अशांत** - Deactivate/Reactivate - **निष्क्रिय करें / पुन: सक्रिय करें** - Search - **खोज करें/ढूंढे** @@ -57,10 +59,11 @@ Zulip friendly and usable. - I want - **मुझे चाहिए** - User - **उपयोगकर्ता** - Person/People - **व्यक्ति/लोग**: "लोग" is the correct plural for - "व्यक्ति", but when talking of *लोग* referring to it as a crowd, we use + "व्यक्ति", but when talking of _लोग_ referring to it as a crowd, we use "भीड़" instead. ## Others(अन्य) + - You - **तुम**: also "आप" if it's in plural. - We - **हम** - Message table - **संदेश बोर्ड** diff --git a/docs/translating/internationalization.md b/docs/translating/internationalization.md index 055d1b22e2..1f25f6e3ee 100644 --- a/docs/translating/internationalization.md +++ b/docs/translating/internationalization.md @@ -272,6 +272,7 @@ If you are passing a translated string to a Handlebars partial, you can use: The syntax for block strings or strings containing variables is: + ```text {{#tr}} Block of English text. @@ -282,7 +283,7 @@ The syntax for block strings or strings containing variables is: {{/tr}} ``` -Just like in JavaScript code, variables are enclosed in *single* +Just like in JavaScript code, variables are enclosed in _single_ braces (rather than the usual Handlebars double braces). Unlike in JavaScript code, variables are automatically escaped by our Handlebars helper. @@ -331,13 +332,12 @@ hostname = https://www.transifex.com This basically identifies you as a Transifex user, so you can access your organizations from the command line. - -[Jinja2]: http://jinja.pocoo.org/ -[Handlebars]: https://handlebarsjs.com/ +[jinja2]: http://jinja.pocoo.org/ +[handlebars]: https://handlebarsjs.com/ [trans]: http://jinja.pocoo.org/docs/dev/templates/#i18n -[FormatJS]: https://formatjs.io/ -[ICU MessageFormat]: https://formatjs.io/docs/intl-messageformat +[formatjs]: https://formatjs.io/ +[icu messageformat]: https://formatjs.io/docs/intl-messageformat [helpers]: https://handlebarsjs.com/guide/block-helpers.html -[Transifex]: https://transifex.com +[transifex]: https://transifex.com [transifexrc]: https://docs.transifex.com/client/client-configuration#transifexrc [html-templates]: ../subsystems/html-css.html#html-templates diff --git a/docs/translating/polish.md b/docs/translating/polish.md index 5ffb37cb54..f57899a620 100644 --- a/docs/translating/polish.md +++ b/docs/translating/polish.md @@ -2,22 +2,22 @@ Use semi-formal Polish for translation, some specifics: -- Informal "you" (*ty*) instead of more formal approaches (e.g. plural - "you" (*wy*), using any formal titles like *Państwo*, *Pan/Pani*). +- Informal "you" (_ty_) instead of more formal approaches (e.g. plural + "you" (_wy_), using any formal titles like _Państwo_, _Pan/Pani_). -- Gender-neutral forms of verbs, e.g. *unsubscribed* - *odsubskrybowano*, - not *odsubskrybowałeś". +- Gender-neutral forms of verbs, e.g. _unsubscribed_ - _odsubskrybowano_, + not \*odsubskrybowałeś". -- Imperative, active and continuous verbs, e.g. *manage streams* - - *zarządzaj kanałami*, not *zarządź kanałami*. +- Imperative, active and continuous verbs, e.g. _manage streams_ - + _zarządzaj kanałami_, not _zarządź kanałami_. -- Not using reflexive *się*, e.g. *log out* would be simply *wyloguj*, - not *wyloguj się*. +- Not using reflexive _się_, e.g. _log out_ would be simply _wyloguj_, + not _wyloguj się_. - Warm and friendly phrasing whenever appropriate. - No slang or regional phrases that could be unclear or too informal, - e.g. *zajawka*. + e.g. _zajawka_. - Consistent usage of Zulip-specific terms and common verbs for actions, even if it means repeating - this is one of the key aspects @@ -40,11 +40,12 @@ Zulip friendly and usable. ## Special terms used in Zulip **alert word**: powiadomienie, "ostrzeżenie" could mean something negative - and alert words in Zulip are used to help users find relevant content +and alert words in Zulip are used to help users find relevant content example: You can set your own alert words for Zulip messages. + > Możesz ustawić powiadomienia dla wybranych fraz w Zulipie. **All messages**: wszystkie wiadomości @@ -52,6 +53,7 @@ You can set your own alert words for Zulip messages. example: You can see all messages in unmuted streams and topics with "All messages". + > Możesz zobaczyć pełną listę wiadomości poprzez widok "Wszystkie wiadomości". **bot**: bot @@ -59,21 +61,24 @@ You can see all messages in unmuted streams and topics with "All messages". example: You can add bots to your Zulip. + > Możesz dodać boty do swojego Zulipa. -**customization**: personalizacja, *kastomizacja* could be too awkward - and *dostosowanie do potrzeb klienta* is too long +**customization**: personalizacja, _kastomizacja_ could be too awkward +and _dostosowanie do potrzeb klienta_ is too long example: You can personalize Zulip in many ways, e.g. by pinning certain streams. + > Możesz spersonalizować Zulipa na wiele sposobów, np. przypinając niektóre kanały. -**emoji**: emoji, both in singular and plural, *ikona emoji* is a pleonasm +**emoji**: emoji, both in singular and plural, _ikona emoji_ is a pleonasm example: Zulip supports emoji both in messages and as reactions. + > Zulip wspiera używanie emoji w wiadomościach i jako reakcje. **filter**: filtr (noun) and filtrowanie (verb) @@ -81,14 +86,16 @@ Zulip supports emoji both in messages and as reactions. example: You can filter the messages by searching for relevant terms. + > Możesz przefiltrować wiadomości poprzez wyszukiwanie. -**group PM**: czat grupowy, different from *wiadomość* since the usage - of *czat grupowy* seems more common +**group PM**: czat grupowy, different from _wiadomość_ since the usage +of _czat grupowy_ seems more common example: You can start a group PM with users in your organization. + > Możesz rozpocząć czat grupowy z użytkownikami w Twojej organizacji. **integration**: integracja @@ -96,6 +103,7 @@ You can start a group PM with users in your organization. example: Zulip supports multiple third-party integrations. + > Zulip wspiera wiele zewnętrznych integracji. **I want**: chcę @@ -103,6 +111,7 @@ Zulip supports multiple third-party integrations. example: I want to change my password. + > Chcę zmienić hasło. **invalid**: niepoprawny/a @@ -110,6 +119,7 @@ I want to change my password. example: Invalid command. + > Niepoprawna instrukcja. **me**: me, no translation since it's used as `/me` @@ -117,6 +127,7 @@ Invalid command. example: You can use `/me` to write a reaction message. + > Możesz napisać wiadomość w formie komentarza poprzez użycie `me`. **mention**: oznaczenie (noun) and oznaczyć (verb) @@ -124,6 +135,7 @@ You can use `/me` to write a reaction message. example: You can mention other Zulip users by using @. + > Możesz oznaczyć innych użytkowników Zulipa używając @. **message**: wiadomość @@ -131,6 +143,7 @@ You can mention other Zulip users by using @. example: You got a new message. + > Masz nową wiadomość. **message table**: lista wiadomości @@ -138,6 +151,7 @@ You got a new message. example: The middle column in Zulip is a message table of all messages in current narrow. + > Środkowa kolumna w Zulipie zawiera listę wiadomości w wybranym widoku. **muting a stream/topic**: wyciszenie kanału/wątku @@ -146,6 +160,7 @@ The middle column in Zulip is a message table of all messages in current narrow. example: You can mute any topic in Zulip through the left-side panel. + > Możesz wyciszyć dowolny wątek w Zulipie używając menu kontekstowego po lewej. **narrow**: widok (noun) and zawęzić (verb) @@ -153,6 +168,7 @@ You can mute any topic in Zulip through the left-side panel. example: You can narrow the messages to any stream, topic or search results. + > Możesz zawęzić wiadomości do wybranego kanału, wątku lub wyników wyszukiwania. **notification**: powiadomienie @@ -160,6 +176,7 @@ You can narrow the messages to any stream, topic or search results. example: Turn on notifications. + > Włącz powiadomienia. **person**: osoba, najczęściej użytkownik @@ -170,6 +187,7 @@ Turn on notifications. example: You can pin streams in the left-side panel. + > Możesz przypiąć wybrane kanały w menu kontekstowym po lewej. **private message**: prywatna wiadomość @@ -177,6 +195,7 @@ You can pin streams in the left-side panel. example: You can send a private message to other users in your organization. + > Możesz wysłać prywatną wiadomość do innych użytkowników w Twojej organizacji. **PM**: PM, translation could be confusing @@ -184,6 +203,7 @@ You can send a private message to other users in your organization. example: Private messages are often abbreviated to PM. + > Prywatne wiadomości są też nazywane PMami, od angielskiego "private message". **private stream**: prywatny kanał @@ -192,6 +212,7 @@ Private messages are often abbreviated to PM. example: Join a private stream. + > Dołącz do prywatnego kanału. **search**: wyszukiwanie (noun) and wyszukaj (verb) @@ -200,8 +221,9 @@ example: Zulip allows you to search messages in all streams and topics you are subscribed to. + > Zulip pozwala na wyszukiwanie wiadomości we wszystkich subskrybowanych kanałach - i wątkach. +> i wątkach. **starred message**: oznaczona wiadomość (noun) **star**: oznaczyć @@ -209,25 +231,29 @@ subscribed to. example: You starred this message. + > Ta wiadomość została oznaczona. -**stream**: kanał, similar to a tv channel - *strumień* sounds a bit artificial +**stream**: kanał, similar to a tv channel - _strumień_ sounds a bit artificial example: You can create new streams in Zulip. + > Możesz tworzyć nowe kanały w Zulipie. **subscribing to a stream**: (za)subskrybowanie kanału (noun) and - (za)subskrybować kanał (verb), perfective form depending on the - context +(za)subskrybować kanał (verb), perfective form depending on the +context examples: Subscribe to a stream. + > Zasubskrybuj kanał. You are not subscribed to this stream. + > Nie subskrybujesz tego kanału. **topic**: wątek @@ -235,14 +261,16 @@ You are not subscribed to this stream. example: Add a new topic. + > Dodaj nowy wątek. **unsubscribing from a stream**: odsubskrybowanie kanału (noun) and - odsubskrybować kanał (verb) +odsubskrybować kanał (verb) example: You have unsubscribed from this stream. + > Odsubskrybowano kanał. **user**: użytkownik @@ -250,6 +278,7 @@ You have unsubscribed from this stream. example: Zulip supports an unlimited number of users in an organization. + > Zulip nie limituje liczby użytkowników w organizacji. **view**: widok, see: **narrow** diff --git a/docs/translating/spanish.md b/docs/translating/spanish.md index f5192a845e..4f12b60840 100644 --- a/docs/translating/spanish.md +++ b/docs/translating/spanish.md @@ -2,15 +2,15 @@ Use informal Spanish for translation: -- Informal "you" (*tú*) instead of formal form *usted*. Many top software +- Informal "you" (_tú_) instead of formal form _usted_. Many top software companies (e.g. Google) use the informal one, because it's much more common in the daily language and avoids making translations look like they were written by machines. -- Imperative, active, and continuous verbs, e.g. *manage streams* - - *gestionar canales*, not *gestión de canales*. +- Imperative, active, and continuous verbs, e.g. _manage streams_ - + _gestionar canales_, not _gestión de canales_. -- Not using reflexive *se* e.g. *log out* should be *salir*, not *salirse*, +- Not using reflexive _se_ e.g. _log out_ should be _salir_, not _salirse_, whenever the infinitive form is possible without making the translation awkward. @@ -32,26 +32,27 @@ with other Spanish speakers in the community. It's all about making Zulip friendly and usable. ## Términos + - Message - **Mensaje** - Private message (PM) - **Mensaje privado (MP)** - Group PM - **mensaje privado grupal**: even though "MP grupal" is the most precise translation, preferably don't use that one. Many users may not - associate "MP" with *private message* in a group context. Better use it + associate "MP" with _private message_ in a group context. Better use it without abbreviations. - Stream - **Canal**: the use of the literal Spanish word for stream "Flujo" is very confusing and not the correct metaphor for Spanish - speaking people. The correct term would be "canal" (*channel*). + speaking people. The correct term would be "canal" (_channel_). - Topic - **Tema** - Private/invite-only stream - **Canal privado/limitado por invitación**: (lit. - *channel limited by invitation*) + _channel limited by invitation_) - Public stream - **Canal público** - Bot - **Bot** - Embedded bot - **Bot integrado** - Interactive bot - **Bot interactivo** - Integration - **Integración** - Notification - **Notificación** -- Alert word - **Alerta**: this is only *alert*. Nonetheless, adding *word* may - make the term confusing (something like *danger!* could be a "palabra de +- Alert word - **Alerta**: this is only _alert_. Nonetheless, adding _word_ may + make the term confusing (something like _danger!_ could be a "palabra de alerta" as well). Google Alerts uses "alerta" in its Spanish translation. - View - **Vista** - Filter - **Filtro**: as used with narrowing (see below). @@ -62,23 +63,25 @@ Zulip friendly and usable. - Endpoint - **Endpoint** ## Frases + - Subscribe/Unsubscribe to a stream - **Suscribir a/Desuscribir de un canal** -- Narrow to - **Filtrar solo**: this is *filter only*, because there's no other - word that's common enough in Spanish for *to narrow* except for "filtrar". +- Narrow to - **Filtrar solo**: this is _filter only_, because there's no other + word that's common enough in Spanish for _to narrow_ except for "filtrar". - Mute/Unmute - **Silenciar/No silenciar** - Deactivate/Reactivate - **Desactivar/Reactivar** - Search - **Buscar** -- Pin - **Fijar** (lit. *to fixate*) +- Pin - **Fijar** (lit. _to fixate_) - Mention/@mention - **Mencionar/@mención** - Invalid - **Inválido** - Customization - **Personalización** - I want - **Yo quiero** - User - **Usuario** - Person/People - **Persona/Personas**: "personas" is the correct plural for - "person", but when talking of *people* referring to it as a crowd, we use + "person", but when talking of _people_ referring to it as a crowd, we use "gente" instead. ## Otros + - You - **Tú**: also "vosotros" if it's in plural. - We - **Nosotros** - Message table - **Tablón de mensajes** diff --git a/docs/translating/translating.md b/docs/translating/translating.md index 1f4d079ab0..610f555693 100644 --- a/docs/translating/translating.md +++ b/docs/translating/translating.md @@ -21,13 +21,13 @@ These are the steps you should follow if you want to help translate Zulip: 1. Join [#translation][translation-stream] in the [Zulip development -community server](../contributing/chat-zulip-org.md), and say hello. -That stream is also the right place for any questions, updates on your -progress, reporting problematic strings, etc. + community server](../contributing/chat-zulip-org.md), and say hello. + That stream is also the right place for any questions, updates on your + progress, reporting problematic strings, etc. 1. Sign up for [Transifex](https://www.transifex.com) and ask to join the [Zulip -project on Transifex](https://www.transifex.com/zulip/zulip/), requesting access -to any languages that you'd like to contribute to (or add new ones). + project on Transifex](https://www.transifex.com/zulip/zulip/), requesting access + to any languages that you'd like to contribute to (or add new ones). ```eval_rst .. note:: @@ -44,6 +44,7 @@ to any languages that you'd like to contribute to (or add new ones). 1. Translate the strings for your language in Transifex. Zulip has several resource files: + - `mobile.json` is for the iOS/Android mobile apps. - `desktop.json` is for the parts of the Zulip desktop apps that are not shared with the Zulip webapp. @@ -90,7 +91,7 @@ Some useful tips for your translating journey: the [Zulip development community server](../contributing/chat-zulip-org.md). - If there are multiple possible translations for a term, search for it in - the *Concordance* tool (the button with a magnet in the top right corner). + the _Concordance_ tool (the button with a magnet in the top right corner). It will show if anyone translated that term before, so we can achieve good consistency with all the translations, no matter who makes them. @@ -109,11 +110,11 @@ if setting one up is a problem for you, ask in chat.zulip.org and we can usually just deploy the latest translations there. - First, download the updated resource files from Transifex using the -`tools/i18n/sync-translations` command (it will require some [initial -setup](../translating/internationalization.html#transifex-cli-setup)). This -command will download the resource files from Transifex and replace -your local resource files with them, and then compile them. You can -now test your translation work in the Zulip UI. + `tools/i18n/sync-translations` command (it will require some [initial + setup](../translating/internationalization.html#transifex-cli-setup)). This + command will download the resource files from Transifex and replace + your local resource files with them, and then compile them. You can + now test your translation work in the Zulip UI. There are a few ways to see your translations in the Zulip UI: @@ -182,15 +183,15 @@ properly capitalized in a way consistent with how Zulip does capitalization in general. This means that: - The first letter of a sentence or phrase should be capitalized. - - Correct: "Manage streams" - - Incorrect: "Manage Streams" + - Correct: "Manage streams" + - Incorrect: "Manage Streams" - All proper nouns should be capitalized. - - Correct: "This is Zulip" - - Incorrect: "This is zulip" + - Correct: "This is Zulip" + - Incorrect: "This is zulip" - All common words like URL, HTTP, etc. should be written in their standard forms. - - Correct: "URL" - - Incorrect: "Url" + - Correct: "URL" + - Incorrect: "Url" The Zulip test suite enforces these capitalization guidelines in the webapp codebase [in our test diff --git a/docs/tutorials/life-of-a-request.md b/docs/tutorials/life-of-a-request.md index 604c9c9177..dce216897d 100644 --- a/docs/tutorials/life-of-a-request.md +++ b/docs/tutorials/life-of-a-request.md @@ -99,7 +99,7 @@ for its API. This means that we use: useful to check a link without downloading a potentially large link - OPTIONS (handled automatically, see more below) -Of these, PUT, DELETE, HEAD, OPTIONS, and GET are *idempotent*, which +Of these, PUT, DELETE, HEAD, OPTIONS, and GET are _idempotent_, which means that we can send the request multiple times and get the same state on the server. You might get a different response after the first request, as we like to give our clients an error so they know that no diff --git a/docs/tutorials/new-feature-tutorial.md b/docs/tutorials/new-feature-tutorial.md index e906dd25be..71e8e501fc 100644 --- a/docs/tutorials/new-feature-tutorial.md +++ b/docs/tutorials/new-feature-tutorial.md @@ -34,6 +34,7 @@ This tutorial will walk through adding a new feature to a Realm (an organization in Zulip). The following files are involved in the process: **Backend** + - `zerver/models.py`: Defines the database model. - `zerver/views/realm.py`: The view function that implements the API endpoint for editing realm objects. @@ -42,20 +43,23 @@ organization in Zulip). The following files are involved in the process: consistent and correct. **Frontend** + - `static/templates/settings/organization_permissions_admin.hbs`: defines - the structure of the admin permissions page (checkboxes for each organization - permission setting). + the structure of the admin permissions page (checkboxes for each organization + permission setting). - `static/js/settings_org.js`: handles organization setting form submission. - `static/js/server_events_dispatch.js`: handles events coming from the server (ex: pushing an organization change to other open browsers and updating the application's state). **Backend testing** + - `zerver/tests/test_realm.py`: end-to-end API tests for updating realm settings. - `zerver/tests/test_events.py`: tests for possible race bugs in the zerver/lib/events.py implementation. **Frontend testing** + - `frontend_tests/puppeteer_tests/admin.ts`: end-to-end tests for the organization admin settings pages. - `frontend_tests/node_tests/dispatch.js` @@ -79,8 +83,7 @@ to learn more about creating and applying database migrations. **Test your changes:** Once you've run the migration, flush memcached on your development server (`./scripts/setup/flush-memcached`) and then -[restart the development server]( -../development/remote.html?highlight=tools%2Frun-dev.py#running-the-development-server) +[restart the development server](../development/remote.html?highlight=tools%2Frun-dev.py#running-the-development-server) to avoid interacting with cached objects. ### Backend changes @@ -152,14 +155,12 @@ documentation Zulip has, see [Documentation](../documentation/overview.md). This example describes the process of adding a new setting to Zulip: a flag that allows an admin to require topics on stream messages (the default behavior is that topics can have no subject). This flag is an -actual Zulip feature. You can review [the original commit]( -https://github.com/zulip/zulip/pull/5660/commits/aeeb81d3ff0e0cc201e891cec07e1d2cd0a2060d) +actual Zulip feature. You can review [the original commit](https://github.com/zulip/zulip/pull/5660/commits/aeeb81d3ff0e0cc201e891cec07e1d2cd0a2060d) in the Zulip repo. (This commit displays the work of setting up a checkbox for the feature on the admin settings page, communicating and saving updates to the setting to the database, and updating the state of the application after the setting is updated. For the code that accomplishes the underlying -task of requiring messages to have a topic, you can [view this commit]( -https://github.com/zulip/zulip/commit/90e2f5053f5958b44ea9b2362cadcb076deaa975).) +task of requiring messages to have a topic, you can [view this commit](https://github.com/zulip/zulip/commit/90e2f5053f5958b44ea9b2362cadcb076deaa975).) ### Update the model @@ -167,7 +168,7 @@ First, update the database and model to store the new setting. Add a new boolean field, `mandatory_topics`, to the Realm model in `zerver/models.py`. -``` diff +```diff # zerver/models.py class Realm(models.Model): @@ -184,7 +185,7 @@ dictionary, where the key is the name of the realm field and the value is the field's type. Add the new field to the `property_types` dictionary. -``` diff +```diff # zerver/models.py class Realm(models.Model) @@ -203,10 +204,10 @@ dictionary. logic and thus cannot use this framework. For example: - The realm `authentication_methods` attribute is a bitfield and needs -additional code for validation and updating. + additional code for validation and updating. - The `allow_message_editing` and `message_content_edit_limit_seconds` -fields depend on one another, so they are also handled separately and -not included in `property_types`. + fields depend on one another, so they are also handled separately and + not included in `property_types`. When creating a realm property that is not a boolean, Text or integer field, or when adding a field that is dependent on other fields, @@ -229,6 +230,7 @@ is helpful. Apply the migration using Django's `migrate` command: `./manage.py migrate`. Output: + ```console shell $ ./manage.py migrate Operations to perform: @@ -244,8 +246,7 @@ Running migrations: ``` Once you've run the migration, restart memcached on your development -server (`/etc/init.d/memcached restart`) and then [restart the development server]( -../development/remote.html?highlight=tools%2Frun-dev.py#running-the-development-server) +server (`/etc/init.d/memcached restart`) and then [restart the development server](../development/remote.html?highlight=tools%2Frun-dev.py#running-the-development-server) to avoid interacting with cached objects. ### Handle database interactions @@ -255,7 +256,7 @@ Like typical apps, we will need our backend to update the database and send some response to the client that made the request. Beyond that, we need to orchestrate notifications about the setting change -to *other* clients (or other users, if you will). Clients +to _other_ clients (or other users, if you will). Clients find out about settings through two closely related code paths. When a client first contacts the server, the server sends the client its initial state. Subsequently, clients subscribe to "events," which can @@ -414,7 +415,7 @@ You'll need to add a parameter for the new field to the `update_realm` function in `zerver/views/realm.py` (and add the appropriate mypy type annotation). -``` diff +```diff # zerver/views/realm.py def update_realm( @@ -516,7 +517,7 @@ template. Then add the new form control in `static/js/admin.js`. -``` diff +```diff // static/js/admin.js function _setup_page() { @@ -536,27 +537,27 @@ The JavaScript code for organization settings and permissions can be found in In frontend, we have split the `property_types` into three objects: - `org_profile`: This contains properties for the "organization - profile" settings page. + profile" settings page. - `org_settings`: This contains properties for the "organization - settings" page. Settings belonging to this section generally - decide what features should be available to a user like deleting a - message, message edit history etc. Our `mandatory_topics` feature - belongs in this section. + settings" page. Settings belonging to this section generally + decide what features should be available to a user like deleting a + message, message edit history etc. Our `mandatory_topics` feature + belongs in this section. - `org_permissions`: This contains properties for the "organization - permissions" section. These properties control security controls - like who can join the organization and whether normal users can - create streams or upload custom emoji. + permissions" section. These properties control security controls + like who can join the organization and whether normal users can + create streams or upload custom emoji. Once you've determined whether the new setting belongs, the next step is to find the right subsection of that page to put the setting in. For example in this case of `mandatory_topics` it will lie in "Other settings" (`other_settings`) subsection. -*If you're not sure in which section your feature belongs, it's +_If you're not sure in which section your feature belongs, it's better to discuss it in the [community](https://chat.zulip.org/) -before implementing it.* +before implementing it._ Note that some settings, like `realm_msg_edit_limit_setting`, require special treatment, because they don't match the common @@ -565,14 +566,14 @@ such input elements with those in `page_params`, so we have to manually handle such situations in a couple key functions: - `settings_org.get_property_value`: This processes the property name - when it doesn't match a corresponding key in `page_params`, and - returns the current value of that property, which we can use to - compare and set the values of corresponding DOM element. + when it doesn't match a corresponding key in `page_params`, and + returns the current value of that property, which we can use to + compare and set the values of corresponding DOM element. - `settings_org.update_dependent_subsettings`: This handles settings - whose value and state depend on other elements. For example, - `realm_waiting_period_threshold` is only shown for with the right - state of `realm_waiting_period_setting`. + whose value and state depend on other elements. For example, + `realm_waiting_period_threshold` is only shown for with the right + state of `realm_waiting_period_setting`. Finally, update `server_events_dispatch.js` to handle related events coming from the server. There is an object, `realm_settings`, in the function @@ -589,7 +590,7 @@ setting has changed, your function should be referenced in the `realm_settings` of `server_events_dispatch.js`. See for example `settings_emoji.update_custom_emoji_ui`. -``` diff +```diff // static/js/server_events_dispatch.js function dispatch_normal_event(event) { diff --git a/docs/tutorials/reading-list.md b/docs/tutorials/reading-list.md index 6ebf581d0a..96a76f5767 100644 --- a/docs/tutorials/reading-list.md +++ b/docs/tutorials/reading-list.md @@ -6,87 +6,86 @@ learning new skills, or polishing the ones you already have. The topics cover a wide variety of topics, from basic Python coding to general developing guidelines. -Feel free to create a pull request in Zulip's [GitHub repository]( -https://github.com/zulip/zulip/) with any interesting books, articles or +Feel free to create a pull request in Zulip's [GitHub repository](https://github.com/zulip/zulip/) with any interesting books, articles or videos you would like to see in this list. Some titles have been shortened for organizational purposes. ## General programming/IT -*Book* - [Clean Code: A Handbook of Agile Software Craftsmanship](https://www.amazon.com/Clean-Code-Handbook-Software-Craftsmanship/dp/0132350882) *(Not free!)* +_Book_ - [Clean Code: A Handbook of Agile Software Craftsmanship](https://www.amazon.com/Clean-Code-Handbook-Software-Craftsmanship/dp/0132350882) _(Not free!)_ -*Books* - [Free programming books list](https://github.com/vhf/free-programming-books) +_Books_ - [Free programming books list](https://github.com/vhf/free-programming-books) -*Blog* - [Free Code Camp blog](https://medium.freecodecamp.com) +_Blog_ - [Free Code Camp blog](https://medium.freecodecamp.com) -*Blog* - [Idle Words talks transcripts](https://idlewords.com/talks) +_Blog_ - [Idle Words talks transcripts](https://idlewords.com/talks) -*Tutorial* - [HTTP Can Do That?!, by Sumana Harihareswara (PyCon 2016)](https://youtu.be/HsLrXt2l-kg) +_Tutorial_ - [HTTP Can Do That?!, by Sumana Harihareswara (PyCon 2016)](https://youtu.be/HsLrXt2l-kg) -*Video* - [Minimum Viable Documentation, by Matthew Lyon (WriteTheDocs 2014)](https://youtu.be/bEZcodengwk) +_Video_ - [Minimum Viable Documentation, by Matthew Lyon (WriteTheDocs 2014)](https://youtu.be/bEZcodengwk) -*Video* - [NoOps, by Kelsey Hightower (DepOpsDays 2016)](https://youtu.be/ajT90pC3ris) +_Video_ - [NoOps, by Kelsey Hightower (DepOpsDays 2016)](https://youtu.be/ajT90pC3ris) -*Video* - [The mind behind Linux (TED interview)](https://youtu.be/o8NPllzkFhE) +_Video_ - [The mind behind Linux (TED interview)](https://youtu.be/o8NPllzkFhE) -*Tutorial* - [Learn code the hard way](https://learncodethehardway.org) +_Tutorial_ - [Learn code the hard way](https://learncodethehardway.org) -*Tutorial* - [What happens when...](https://github.com/alex/what-happens-when) +_Tutorial_ - [What happens when...](https://github.com/alex/what-happens-when) -*Article* - [An Interview With Linus Torvalds](https://techcrunch.com/2012/04/19/an-interview-with-millenium-technology-prize-finalist-linus-torvalds) +_Article_ - [An Interview With Linus Torvalds](https://techcrunch.com/2012/04/19/an-interview-with-millenium-technology-prize-finalist-linus-torvalds) -*Article* - [Effective Learning Strategies for Programmers](https://akaptur.com/blog/2015/10/10/effective-learning-strategies-for-programmers/) +_Article_ - [Effective Learning Strategies for Programmers](https://akaptur.com/blog/2015/10/10/effective-learning-strategies-for-programmers/) -*Article* - [Readme Driven Development](https://tom.preston-werner.com/2010/08/23/readme-driven-development.html) +_Article_ - [Readme Driven Development](https://tom.preston-werner.com/2010/08/23/readme-driven-development.html) -*Article* - [Systematic Debugging](https://akaptur.com/blog/2013/07/24/systematic-debugging) +_Article_ - [Systematic Debugging](https://akaptur.com/blog/2013/07/24/systematic-debugging) -*Paper* - [Floating-Point Arithmetic](https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html) +_Paper_ - [Floating-Point Arithmetic](https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html) ## Python -*Video* - [Intro to Python for beginners, by Jessica McKellar (PyCon 2013)](https://youtu.be/rkx5_MRAV3A) +_Video_ - [Intro to Python for beginners, by Jessica McKellar (PyCon 2013)](https://youtu.be/rkx5_MRAV3A) -*Video* - [Breaking the rules, by Jessica McKellar (PyCon Sweden)](https://youtu.be/C0fnHhY9UOc) +_Video_ - [Breaking the rules, by Jessica McKellar (PyCon Sweden)](https://youtu.be/C0fnHhY9UOc) -*Video* - [Build & break a Python sandbox, by Jessica McKellar (PyCon 2014)](https://pyvideo.org/pycon-us-2014/building-and-breaking-a-python-sandbox.html) +_Video_ - [Build & break a Python sandbox, by Jessica McKellar (PyCon 2014)](https://pyvideo.org/pycon-us-2014/building-and-breaking-a-python-sandbox.html) -*Video* - [Cache me if you can, by Guillaume Ardaud (PyCon 2014)](https://pyvideo.org/pycon-us-2014/cache-me-if-you-can-memcached-caching-patterns.html) +_Video_ - [Cache me if you can, by Guillaume Ardaud (PyCon 2014)](https://pyvideo.org/pycon-us-2014/cache-me-if-you-can-memcached-caching-patterns.html) -*Video* - [Loop like a native, by Ned Batchelder (PyCon 2013)](https://youtu.be/EnSu9hHGq5o) +_Video_ - [Loop like a native, by Ned Batchelder (PyCon 2013)](https://youtu.be/EnSu9hHGq5o) -*Video* - [Modern Dictionaries, by Raymond Hettinger (SF Python)](https://youtu.be/p33CVV29OG8) +_Video_ - [Modern Dictionaries, by Raymond Hettinger (SF Python)](https://youtu.be/p33CVV29OG8) -*Video* - [Python Language, by Guido van Rossum (PyCon 2016)](https://youtu.be/YgtL4S7Hrwo) +_Video_ - [Python Language, by Guido van Rossum (PyCon 2016)](https://youtu.be/YgtL4S7Hrwo) -*Video* - [The Mighty Dictionary, by Brandon Rhodes (PyCon 2010)](https://pyvideo.org/pycon-us-2010/the-mighty-dictionary-55.html) +_Video_ - [The Mighty Dictionary, by Brandon Rhodes (PyCon 2010)](https://pyvideo.org/pycon-us-2010/the-mighty-dictionary-55.html) -*Article* - [Static types in Python, oh my(py)!](https://blog.zulip.org/2016/10/13/static-types-in-python-oh-mypy) +_Article_ - [Static types in Python, oh my(py)!](https://blog.zulip.org/2016/10/13/static-types-in-python-oh-mypy) -*Guide* - [The Hitchhiker’s Guide to Python!](https://docs.python-guide.org/) +_Guide_ - [The Hitchhiker’s Guide to Python!](https://docs.python-guide.org/) ## Java/Android -*Course* - [Android Development for Beginners](https://www.udacity.com/course/android-development-for-beginners--ud837) +_Course_ - [Android Development for Beginners](https://www.udacity.com/course/android-development-for-beginners--ud837) -*Blog* - [Java Tutorials for Beginners](https://www.geeksforgeeks.org/java/) +_Blog_ - [Java Tutorials for Beginners](https://www.geeksforgeeks.org/java/) ## JavaScript/ECMAScript -*Tutorial* - [clean-code-javascript Software engineering principles](https://github.com/ryanmcdermott/clean-code-javascript) +_Tutorial_ - [clean-code-javascript Software engineering principles](https://github.com/ryanmcdermott/clean-code-javascript) -*Course* - [React native and redux course](https://www.udemy.com/course/the-complete-react-native-and-redux-course/) (*Not free!*) +_Course_ - [React native and redux course](https://www.udemy.com/course/the-complete-react-native-and-redux-course/) (_Not free!_) -*Slides* - [TypeScript vs. CoffeeScript vs. ES6](https://www.slideshare.net/NeilGreen1/type-script-vs-coffeescript-vs-es6) +_Slides_ - [TypeScript vs. CoffeeScript vs. ES6](https://www.slideshare.net/NeilGreen1/type-script-vs-coffeescript-vs-es6) ## TypeScript -*Tutorial* - [TypeScript handbook section on base types](https://www.typescriptlang.org/docs/handbook/basic-types.html) +_Tutorial_ - [TypeScript handbook section on base types](https://www.typescriptlang.org/docs/handbook/basic-types.html) -*Book* - [TypeScript Deep Dive](https://basarat.gitbooks.io/typescript/) +_Book_ - [TypeScript Deep Dive](https://basarat.gitbooks.io/typescript/) -*Guide* - [TypeScript Declaration Files Introduction](https://www.typescriptlang.org/docs/handbook/declaration-files/introduction.html) +_Guide_ - [TypeScript Declaration Files Introduction](https://www.typescriptlang.org/docs/handbook/declaration-files/introduction.html) ## Git/version control systems (VCS) @@ -94,31 +93,31 @@ You may want to take a look first at our [Git and GitHub guide](../git/index.md) ## Computer science/algorithms -*Blog* - [GeeksforGeeks](https://www.geeksforgeeks.org) +_Blog_ - [GeeksforGeeks](https://www.geeksforgeeks.org) -*Book* [Introduction to Algorithms](https://mitpress.mit.edu/books/introduction-algorithms) (*Not free!*) +_Book_ [Introduction to Algorithms](https://mitpress.mit.edu/books/introduction-algorithms) (_Not free!_) -*Blog* - [Setosa data visualization and visual explanations](https://setosa.io) +_Blog_ - [Setosa data visualization and visual explanations](https://setosa.io) -*Course* - [Algorithms, Part I](https://www.coursera.org/learn/algorithms-part1) +_Course_ - [Algorithms, Part I](https://www.coursera.org/learn/algorithms-part1) -*Course* - [Open Source Society University](https://ossu.firebaseapp.com) +_Course_ - [Open Source Society University](https://ossu.firebaseapp.com) -*Course* - [MIT CSAIL 6.828: Operative Systems Engineering](https://pdos.csail.mit.edu/6.828/2016) +_Course_ - [MIT CSAIL 6.828: Operative Systems Engineering](https://pdos.csail.mit.edu/6.828/2016) ## Community experience -*Book* - [Producing Open Source Software](https://producingoss.com/en/) +_Book_ - [Producing Open Source Software](https://producingoss.com/en/) -*Article* - [Advice on Starting And Running A New Open Source Project](https://www.harihareswara.net/sumana/2016/08/04/1) +_Article_ - [Advice on Starting And Running A New Open Source Project](https://www.harihareswara.net/sumana/2016/08/04/1) -*Article* - [How to ask good questions](https://jvns.ca/blog/good-questions) +_Article_ - [How to ask good questions](https://jvns.ca/blog/good-questions) -*Article* - [Notes for New FLOSS Contributors](https://www.harihareswara.net/sumana/2016/10/12/0) +_Article_ - [Notes for New FLOSS Contributors](https://www.harihareswara.net/sumana/2016/10/12/0) -*Article* - [To be mentored](https://trueskawka.github.io/zulip/outreachy/blog/2017/01/02/to-be-mentored.html) +_Article_ - [To be mentored](https://trueskawka.github.io/zulip/outreachy/blog/2017/01/02/to-be-mentored.html) -*Article* - [To mentor](https://trueskawka.github.io/zulip/outreachy/gci/blog/2017/01/03/to-mentor.html) +_Article_ - [To mentor](https://trueskawka.github.io/zulip/outreachy/gci/blog/2017/01/03/to-mentor.html) [List of good projects for new contributors](https://github.com/MunGell/awesome-for-beginners) diff --git a/docs/tutorials/screenshot-and-gif-software.md b/docs/tutorials/screenshot-and-gif-software.md index b0bcc2588e..1157e97b36 100644 --- a/docs/tutorials/screenshot-and-gif-software.md +++ b/docs/tutorials/screenshot-and-gif-software.md @@ -6,13 +6,16 @@ requests, as other contributors can see the changes you have made without having to check out your branch. ## Screenshot tools by platform + ### Browser + - Firefox can take screenshots without any plugins (stable feature starting from v57) - - You can find it under `page actions / Take a screenshot` - - It is capable of screenshotting the entire page, visible area and individual DOM elements + - You can find it under `page actions / Take a screenshot` + - It is capable of screenshotting the entire page, visible area and individual DOM elements - [LightShot Screenshot](https://app.prntscr.com/en/index.html) (Chrome, Firefox, IE & Opera) ### macOS + - Command-Shift-3 to capture the entire screen - Command-Shift-4 and drag cursor to select a specific area - Command-Shift-5 and select suitable option to capture or record screen (macOS Mojave) @@ -20,6 +23,7 @@ to check out your branch. - [Gyazo](https://gyazo.com/en) ### Windows + - PrtScn to copy the screen to the clipboard - Windows-PrtScn to save the screen to a file - Snipping Tool (inbuilt) @@ -27,15 +31,19 @@ to check out your branch. - [Gyazo](https://gyazo.com/en) ### Linux + - gnome-screenshot (inbuilt, you can use Ctrl-Shift-PrtScn as a shortcut for its “select an area to grab” feature) ## GIF tools by platform + ### Browser + - [GIPHY](https://giphy.com) - [Chrome Capture](https://chrome.google.com/webstore/detail/chrome-capture/ggaabchcecdbomdcnbahdfddfikjmphe?hl=en) -(Tip: Use Alt+i to interact with the page while recording) + (Tip: Use Alt+i to interact with the page while recording) ### macOS + - [QuickTime](https://support.apple.com/en-in/HT201066) - [GIPHY](https://giphy.com/apps/giphycapture) - [CloudApp](https://www.getcloudapp.com) @@ -45,10 +53,12 @@ to check out your branch. - [Gyazo GIF](https://gyazo.com/en) ### Windows + - [ScreenToGif](https://www.screentogif.com) - [Gyazo GIF](https://gyazo.com/en) - [Monosnap](https://www.monosnap.com/welcome) ### Linux + - [Peek](https://github.com/phw/peek) - [SilentCast](https://github.com/colinkeenan/silentcast) diff --git a/docs/tutorials/shell-tips.md b/docs/tutorials/shell-tips.md index 636a0b4f68..85df4290b3 100644 --- a/docs/tutorials/shell-tips.md +++ b/docs/tutorials/shell-tips.md @@ -1,7 +1,7 @@ # Shell tips -The *shell* is a **command line interpreter**. To use it you can open a -*terminal* (sometimes called a *console*). This is how most terminal windows +The _shell_ is a **command line interpreter**. To use it you can open a +_terminal_ (sometimes called a _console_). This is how most terminal windows look like: ![An example shell window](../images/shell-screenshot.png) @@ -26,12 +26,12 @@ awaiting new orders. The prompt can contain useful information, let's look at `(venv)john@laptop:~$`: - `(venv)` informs the user that they're currently in a virtual environment -(more on [Python virtual + (more on [Python virtual environments](https://docs.python-guide.org/dev/virtualenvs/)) - the `john` before `@` is the username - the `laptop` is the host machine name - the `~` after the colon informs the user they're currently in the home -folder of the user `john` + folder of the user `john` You shouldn't type the prompt or the text preceding it, since it isn't a part of the commands. @@ -89,7 +89,7 @@ shell that the following command must be run as the root - a user that by default has access to all commands and files on a Unix operating system (i.e. a user with administrator privileges). That's why you may be asked for a password in those cases: the system verifies you have permission to act as -the *root* user. +the _root_ user. In case you were wondering, the name `sudo` comes from **s**uper **u**ser **do**. @@ -211,9 +211,9 @@ $ cd /home/john/notes Here, the command is `cd`, and the first (and only) argument is `/home/john/notes`: -- `cd` - *command*: changes your current directory. +- `cd` - _command_: changes your current directory. -- `/home/john/notes` - *argument*: the directory where you want to go. +- `/home/john/notes` - _argument_: the directory where you want to go. In each command the arguments are specified in different ways, and have different meanings. @@ -232,13 +232,13 @@ In this case, we're saying: "Bash, use the app `nano` to open the file `todo.txt`, enabling mouse support, and saving the backup files to `/home/john/backups`". The different parts are: -- `nano` - *command*: program that allows editing text easily. +- `nano` - _command_: program that allows editing text easily. -- `-C` - *argument*: needs you to indicate where the backups should be stored, +- `-C` - _argument_: needs you to indicate where the backups should be stored, and thus you have to add an additional argument after it, to specify the directory (`/home/john/backups` in the example). -- `--mouse` - *argument*: is just an option you set, `nano` doesn't need +- `--mouse` - _argument_: is just an option you set, `nano` doesn't need anything else to make it work. Thus, there isn't any extra argument for that. Note that the `todo.txt` is the file we want to open! It has nothing to do with diff --git a/docs/tutorials/writing-views.md b/docs/tutorials/writing-views.md index a461cc85c0..f0d37fc483 100644 --- a/docs/tutorials/writing-views.md +++ b/docs/tutorials/writing-views.md @@ -100,7 +100,6 @@ def home(request: HttpRequest) -> HttpResponse: Templates for the main website are found in [templates/zerver/app](https://github.com/zulip/zulip/tree/main/templates/zerver/app). - ## Writing API REST endpoints These are code-parseable views that take x-www-form-urlencoded or JSON @@ -114,7 +113,7 @@ This method will authenticate the user either through a session token from a cookie on the browser, or from a base64 encoded `email:api-key` string given via HTTP basic auth for API clients. -``` py +```py >>> import requests >>> r = requests.get('https://api.github.com/user', auth=('hello@example.com', '0123456789abcdeFGHIJKLmnopQRSTUV')) >>> r.status_code @@ -146,7 +145,7 @@ arguments for. Here's an example: -``` py +```py from zerver.decorator import require_realm_admin from zerver.lib.request import has_request_variables, REQ @@ -232,7 +231,7 @@ If you're removing data, use DELETE. When writing a new API endpoint, with the exception of things like sending messages, requests should be safe to repeat, without impacting -the state of the server. This is *idempotency*. +the state of the server. This is _idempotency_. You will often want to return an error if a request to change something would do nothing because the state is already as desired, to @@ -343,7 +342,7 @@ If the webhook does not have an option to provide a bot email, use the `webhook_view` decorator, to fill in the `user_profile` and `request.client` fields of a request: -``` py +```py @webhook_view('PagerDuty') @has_request_variables def api_pagerduty_webhook(request, user_profile, @@ -351,6 +350,7 @@ def api_pagerduty_webhook(request, user_profile, stream=REQ(default='pagerduty'), topic=REQ(default=None)): ``` + `request.client` will be the result of `get_client("ZulipPagerDutyWebhook")` in this example and it will be passed to `check_send_stream_message`. For more information, see [Clients in Zulip](../subsystems/client.md). diff --git a/static/shared/README.md b/static/shared/README.md index 73ae7aea33..ec44bf0495 100644 --- a/static/shared/README.md +++ b/static/shared/README.md @@ -3,21 +3,21 @@ and are also incorporated by the Zulip mobile app. Note that the deployment cycles are different: - - In the webapp, this code is deployed in the same way as the rest of - the web frontend: it's part of the server tree, and the browser - gets it from the server, so the client is always running the same - version the server just gave it. +- In the webapp, this code is deployed in the same way as the rest of + the web frontend: it's part of the server tree, and the browser + gets it from the server, so the client is always running the same + version the server just gave it. - - In the mobile app, this code is deployed in the same way as the - rest of the mobile app: it's bundled up into the app binary which - is uploaded to app stores and users install on their devices. The - client will be running the version built into their version of the - mobile app, which may be newer, older, or simply different from the - version on the server. +- In the mobile app, this code is deployed in the same way as the + rest of the mobile app: it's bundled up into the app binary which + is uploaded to app stores and users install on their devices. The + client will be running the version built into their version of the + mobile app, which may be newer, older, or simply different from the + version on the server. - The mobile app always refers to a specific version of this code; - changes to this code will appear in the mobile app only after a - commit in the mobile app pulls them in. + The mobile app always refers to a specific version of this code; + changes to this code will appear in the mobile app only after a + commit in the mobile app pulls them in. To update the version of @zulip/shared on NPM, see the [instructions][publishing-shared] in the mobile repo. diff --git a/tools/droplets/README.md b/tools/droplets/README.md index c31d91be2c..7afd7efe63 100644 --- a/tools/droplets/README.md +++ b/tools/droplets/README.md @@ -60,6 +60,7 @@ $ python3 create.py $ python3 create.py --tags $ python3 create.py --tags ``` + Assigning tags to droplets like `GCI` can be later useful for listing all the droplets created during GCI. [Tags](https://www.digitalocean.com/community/tutorials/how-to-tag-digitalocean-droplets) @@ -75,6 +76,7 @@ user. If you want to recreate a droplet for a user you can pass the ``` $ python3 create.py --recreate ``` + This will destroy the old droplet and create a new droplet for the user. @@ -147,6 +149,7 @@ Rough steps: 1. Open a PR with the updated template_id in zulip/zulip! ## Creating a new base image + Creating a new base image happens rarely since updating the base image is good enough most of the time. Check out https://chat.zulip.org/#narrow/stream/3-backend/topic/new.20base.20dev.20droplet to view the discussion when we attempted to do upgrade last time. @@ -211,7 +214,6 @@ They can remove your SSH keys by running: $ python3 ~/zulip/tools/droplets/add_mentor.py --remove ``` - # Creating a production droplet `create.py` can also create a production droplet quickly for testing purposes. diff --git a/tools/oneclickapps/README.md b/tools/oneclickapps/README.md index 608d5d1f77..da4e0d414e 100644 --- a/tools/oneclickapps/README.md +++ b/tools/oneclickapps/README.md @@ -3,6 +3,7 @@ This directory contains scripts for automating the release of Zulip one click apps. ## DigitalOcean 1-Click Application + `prepare_digital_ocean_one_click_app_release.py` creates the image of DigitalOcean 1-Click app from the latest Zulip release (fetched from https://www.zulip.org/dist/releases). It will also create a test droplet from the image and send the image and droplet @@ -23,11 +24,13 @@ work correctly. These secrets are passed as environment variables to the GitHub - `ONE_CLICK_ACTION_ZULIP_BOT_EMAIL` - The email of the Zulip bot. Also pass the following as environment variables in `.github/workflows/update-oneclick-apps.yml`. -- `PYTHON_DIGITALOCEAN_REQUEST_TIMEOUT_SEC` - This configures the maximum number of seconds + +- `PYTHON_DIGITALOCEAN_REQUEST_TIMEOUT_SEC` - This configures the maximum number of seconds to wait before the requests made by `python-digitalocean` time out. If not configured, it's common for the requests to take 20+ minutes before getting timed out. ### Verifying the one click app image + - The action will send the image name and test droplet details to the stream configured in the above steps. - SSH into the test droplet by following the instructions in the message. @@ -37,6 +40,7 @@ Also pass the following as environment variables in `.github/workflows/update-on This script checks whether the image created is valid. It is also run by the DigitalOcean team before they approve the image submission in the one click app marketplace. + - If there are no errors (you can ignore most of the warnings), exit the SSH connection and reconnect. - Populate the details asked by the installer and verify that the installer completes successfully. @@ -62,6 +66,7 @@ Also pass the following as environment variables in `.github/workflows/update-on **Errors** If there are any errors while setting up the one click app installer, you have three options + - Include the fix in the Fabric script that setups the installer. [01-initial-setup](https://raw.githubusercontent.com/zulip/marketplace-partners/master/marketplace_docs/templates/Fabric/scripts/01-initial-setup) file should be a good place to include the fix. See diff --git a/zilencer/README.md b/zilencer/README.md index e1da509a9d..7296111fef 100644 --- a/zilencer/README.md +++ b/zilencer/README.md @@ -1,5 +1,4 @@ -ZILENCER -- The Zulip license manager -======== +# ZILENCER -- The Zulip license manager This app is the place for storing state about various deployments of Zulip that exist in the world.