Commit Graph

46883 Commits

Author SHA1 Message Date
Austin Riba
27494b8b50 typeahead: Add Flow types for get_emoji_matcher and sort_emojis.
This will let us use these in mobile.
2022-04-04 18:11:17 -07:00
Dinesh
fa563c34c8 templates: Remove unnecessary modal-holders.
e6e6010, 41ddf29 and other commits migrating components to Micromodal
have removed references to these divs. Did a global search and
removed these divs.
2022-04-04 17:55:43 -07:00
Lauryn Menard
460968179c api_docs: Add roles and permissions page to API documentation.
Adds a non-endpoint specific page to the API documentation about
organization-level roles and permissions for users in order to
highlight important and useful information for clients and API
users.

Also, adds links to new documentation page in related areas
of the API documentation.
2022-04-04 17:09:05 -07:00
Greg Price
8bc2ac4b8d shared: Bump version to 0.0.11. shared-0.0.11 2022-04-04 15:39:23 -07:00
Greg Price
d1c326a6cd poll_data: Write down types for Flow, for mobile.
These are based on my reading of the poll_data.js implementation.
2022-04-04 15:37:42 -07:00
Alex Vandiver
5c8086bf90 docs: Fix typo.
We don't suggest self-hosing, unless via a sprinkler in warm weather.
2022-04-04 14:52:04 -07:00
Steve Howell
17b60efdc7 markdown: Inject function for emoticon translations.
We want our parser to be as re-entrant as possible.
2022-04-04 14:07:18 -07:00
Steve Howell
03c15c8c14 markdown: Rename r to renderer. 2022-04-04 14:07:18 -07:00
Steve Howell
214ec099bb markdown: Eliminate setup() call.
It has always been pretty arbitrary what we did inside
of setup() vs. parse(), and we want to avoid unpredictable
results from other platforms neglecting to call setup().

On my machine you can parse a simple message in about
25 microseconds, based on a trial of a million messages
with the content of "**bold**".  Whatever portion of
that time is related to setup-related things like
compiling regexes should be negligible from the user's
perspective, since we never run parse() in a loop.
2022-04-04 14:07:18 -07:00
Steve Howell
093eba077a markdown: Avoid needless code duplication.
We only need to loop through the preprocessors
once, and we should use the options passed
in to the parser, not the default options
from the original setOptions call.

The first loop here was doing nothing.
2022-04-04 14:07:18 -07:00
Steve Howell
a77bf90601 markdown: Narrow stream/user_group types for mobile.
Our sub (i.e stream) and user_group objects have a bunch
of fields that aren't relevant to markdown parsing, so
we create narrow types that make it easier for us to
share code with mobile in the future.

I considered working purely in id space, but the problem
there is that user-entered stream names and user group
names need to be canonicalized.
2022-04-04 14:07:18 -07:00
Steve Howell
2bfdbbe7dc markdown: Extract get_topic_links. 2022-04-04 14:07:18 -07:00
Steve Howell
326dbfb934 markdown: Use options, not rules, for linkifier regexes.
This avoids the need to set a global from linkifiers.js.
2022-04-04 14:07:18 -07:00
Steve Howell
029b3e79a9 markdown: Add abstract_map() concept.
The abstract_map() helper clarifies that our code
doesn't need a concrete Map object from JS. This
change is possibly premature until we get a little
bit closer on integrating with mobile, since it
solves kinda the same problem that we might handle
more elegantly with TypeScript or Flow.

OTOH I find it to be pretty non-intrusive for the
webapp.
2022-04-04 14:07:18 -07:00
Steve Howell
3884710033 markdown: Inject linkifiers helper. 2022-04-04 14:07:18 -07:00
Steve Howell
1156001840 markdown: Inject emoji helpers.
Note that we try to avoid the helpers global, but we
still need a future commit to further clean things up.
2022-04-04 14:07:18 -07:00
Steve Howell
94f1fe6891 markdown: Avoid helpers global in some places.
These are the low-hanging-fruit places where we
can avoid using the helpers global.

The long term goal here is to make the markdown
code truly re-entrant, but some challenges still
remain.
2022-04-04 14:07:18 -07:00
Steve Howell
cc51e89730 markdown: Set handlers at parse time.
This will help us make the code a bit more
re-entrant in future commits.
2022-04-04 14:07:18 -07:00
Steve Howell
38a3d89a13 zcommands: Fully parse messages from the server.
Before this change, we would use **some** options relating
to parsing messages, but not all of them.  The reason for
this was completely unintentional.

It's mostly a moot point, since the server sends back pretty
generic messages when you do something like invoke the
"/dark" command, and the message would parse the same way
whether or not the parser was looking for things like user
mentions or stream links.

In order to make this code predictable, I had to decide
whether we do a completely vanilla parse or a full message
parse. My decision now is mostly tactical. It's a trivial
one-line change to just use all the options for message
parsing, whereas it requires a major overhaul to allow a
vanilla parse.

I also predict that we will eventually want to parse these
server responses as if they were messages. I doubt the
zcommand responses would ever take advantage of it, but I
could imagine things like nag messages wanting to use user
mentions.

Even if my predictions are wrong, my decisions here are
pretty easy to reverse once we learn more.

For the particular case of zcommands, it is puzzling to me
why the server doesn't just send back HTML, but I don't want
to open that can of worms yet, as that would technically be
an API change.

For now I am happy with the one-line fix.
2022-04-04 14:07:18 -07:00
Steve Howell
06ba05b44d markdown: Extract parse_non_message().
The zcommand code was calling directly into the "marked"
library, which was extremely misleading, since you don't
get a vanilla parse of the markdown due to the fact
that markdown.js calls setOptions at initialize time.

This commit shifts the responsibility to markdown.js
as well as adding a bit of test coverage, but it is
otherwise just a pure code-move refactoring.

The next commit will tweak things further.
2022-04-04 14:07:18 -07:00
Steve Howell
cf1149539e emojis: Swap the loops to build the emojis map.
This is mostly done for correctness reasons--it is
easiest from a logical standpoint to set the realm
emojis at the end of the function, since we do not want
them to be overwritten by normal emojis. The code
worked before this change, but it involved a clunky
check to map.has().

There is also probably a very minor performance
improvement insofar as N (the number of normal
emojis) is typically greater than R (the number
of realm emojis), and we eliminate N calls to
map.has in return for R calls to map.set.  Even
if R is quite large, the readability advantages
probably far outweigh any performance considerations,
since we are using native map calls.

Thanks to Austin Riba for this suggestion.
2022-04-04 13:28:49 -07:00
Steve Howell
3f6d0939fc emojis: Extract build_default_emoji_aliases(). 2022-04-04 13:28:49 -07:00
Steve Howell
e7738981b7 emojis: Make build_emojis_by_name() a pure function. 2022-04-04 13:28:49 -07:00
Steve Howell
d10e7ef85c emojis: Make build_emoticon_translations() a pure function. 2022-04-04 13:28:49 -07:00
Steve Howell
c943447c6e emojis: Un-share the emoji.js module.
The mobile app was never able to use the shared
version of emoji.js, because, among other problems
with our code organization, the emoji.js module
is strongly based on a mutate-the-data paradigm
that doesn't play nice with React.  The way
that we mutate data and violate encapsuation
here is something that we would mostly want to fix
without even trying to shared code with mobile, so
subsequent commits will try to extract some pure
functions into a shared module.
2022-04-04 13:28:49 -07:00
Heidi Ahlberg
cf1f70e3ef docs: Add translation guidelines for Finnish. 2022-04-04 13:24:22 -07:00
Archisman Das
491b1513eb settings: Fix length of custom profile field URL input.
The backend validates that URL inputs are RFC valid URLs (with no
specific length limit), but the frontend appears to have a maximum
length of 50 specified, likely because of a copy-paste error.

Increase the HTML maxlength for this input to 2048, which is a length
supported for URLs by all major browsers.

Fixes #21633
2022-04-04 12:10:05 -07:00
Tim Abbott
2ad60b0cda version: Update link to blog post for 5.0 release. 2022-04-04 11:58:10 -07:00
Sahil Batra
c3efd6b6a4 i18n: Translate the whole text in stream deactivation modal.
This commit fixes the template of stream deactivation modal
to tag all the text for translation. This commit also removes
the unnecessary span element.
2022-04-04 11:57:19 -07:00
Aman Agrawal
dcdf071751 message_edit: Fix false sub/unsub bookend on using a near link.
We were not setting the `historical` flag correctly for
messages fetched via `json_fetch_raw_message` when used didn't
have any UserMessage.

Extended relevant tests to fetch check message flags too.
2022-04-04 11:51:12 -07:00
Lauryn Menard
0b03275329 message_send: Move sender parameter to REQ.
Instead of using request.POST to get any potential `sender`
parameters for `send_message_backend`, moves it to the REQ
framework parameters as `req_sender`.

Also, updates `create_mirrored_message_users` to take specific
parameters instead of an HTTP request parameter, which then
accessed parameters via request.POST. And updates existing tests
for those changes.
2022-04-04 11:47:52 -07:00
Alex Vandiver
104e11c4fd version: Update version and changelog after 5.1 release. 2022-04-01 23:17:11 -07:00
Alex Vandiver
35e27aef4a migrations: Remove the possibly-duplicated emoji re-uploading.
In 85e531e377, we duplicated this block
of migration code to fix a bug, but moving it (aka deleting the
original copy) is a cleaner solution.
2022-04-01 17:51:00 -07:00
Alex Vandiver
eb31681934 check-database-compatibility: Ignore squashed and renamed migrations.
Fixes: #21596.
2022-04-01 16:15:41 -07:00
Tim Abbott
85e531e377 migrations: Repeat part of migration 0376.
The blockquote explains the motivation for this change in detail.

Fixes #21608.
2022-04-01 15:20:43 -07:00
NerdyLucifer
c79849dab6 settings: Replace "Delete bot" with "Deactivate bot".
The feature deactivates the bot user; Zulip has no "delete bot"
feature. So fix the label to match what it does.

We also change the icon to match the one we use for deactivating users
in the "Manage users" UI.
2022-04-01 15:03:23 -07:00
Sahil Batra
3a3ed78fd9 stream_settings: Live update the banner on changing subscription. 2022-04-01 14:52:06 -07:00
Palash
4d44698805 stream_settings: Remove pencil icon from 'General' tab in stream settings.
For user who is not an administrator.
Also implemented a banner that notifies the user if they can edit
the following settings (name/description and stream permission).
Also increased padding-top of stream header by 10px. This change is done
to increase vertical spacing between the banner
and the stream header.

Fixes #20001.
2022-04-01 14:52:06 -07:00
Alex Vandiver
0af00a3233 upgrade: Mark puppet as having started the server.
We previously used restart-server if puppet was run, as a nod to the
fact that `supervisor reread && supervisor update` will _start_
service groups that were modified, even if they were previously
stopped; this is because they are marked as `autostart=true`, which is
honored on service change.

However, upgrades want to run while there are no services running.  If
puppet is run, explicitly set the server as potentially being "up", so
that a `shutdown_server()` before migrations, if they exist, will stop
services.
2022-03-31 17:21:39 -07:00
Alex Vandiver
e9596637e7 upgrade: Move the shutdown_server calls to where they are relevant.
shutdown_server is a noop if the server is already stopped; placing
these in each block makes the logic more apparent.
2022-03-31 17:21:39 -07:00
Aman Agrawal
2df4ace441 navbar_alerts: Adjust height of recent topics when alert is visible.
Fixes #21619

We need to adjust height of recent topics along with the app
otherwise the container becomes separately scrollable due to
it overflowing the app height.
2022-03-31 11:25:09 -07:00
Alex Vandiver
2e50ead9d1 data_import: Fix bot email address de-duplication.
4815f6e28b tried to de-duplicate bot
email addresses, but instead caused duplicates to crash:

```
Traceback (most recent call last):
  File "./manage.py", line 157, in <module>
    execute_from_command_line(sys.argv)
  File "./manage.py", line 122, in execute_from_command_line
    utility.execute()
  File "/srv/zulip-venv-cache/56ac6adf406011a100282dd526d03537be84d23e/zulip-py3-venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/srv/zulip-venv-cache/56ac6adf406011a100282dd526d03537be84d23e/zulip-py3-venv/lib/python3.8/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/srv/zulip-venv-cache/56ac6adf406011a100282dd526d03537be84d23e/zulip-py3-venv/lib/python3.8/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/home/zulip/deployments/2022-03-16-22-25-42/zerver/management/commands/convert_slack_data.py", line 59, in handle
    do_convert_data(path, output_dir, token, threads=num_threads)
  File "/home/zulip/deployments/2022-03-16-22-25-42/zerver/data_import/slack.py", line 1320, in do_convert_data
    ) = slack_workspace_to_realm(
  File "/home/zulip/deployments/2022-03-16-22-25-42/zerver/data_import/slack.py", line 141, in slack_workspace_to_realm
    ) = users_to_zerver_userprofile(slack_data_dir, user_list, realm_id, int(NOW), domain_name)
  File "/home/zulip/deployments/2022-03-16-22-25-42/zerver/data_import/slack.py", line 248, in users_to_zerver_userprofile
    email = get_user_email(user, domain_name)
  File "/home/zulip/deployments/2022-03-16-22-25-42/zerver/data_import/slack.py", line 406, in get_user_email
    return SlackBotEmail.get_email(user["profile"], domain_name)
  File "/home/zulip/deployments/2022-03-16-22-25-42/zerver/data_import/slack.py", line 85, in get_email
    email_prefix += cls.duplicate_email_count[email]
TypeError: can only concatenate str (not "int") to str
```

Fix the stringification, make it case-insensitive, append with a dash
for readability, and add tests for all of the above.
2022-03-31 11:10:18 -07:00
Alex Vandiver
65e19c4fbd supervisor: 'foo:*' also matches 'foo'.
7c4293a7d3 switched to checking if the
service was already running, and use `supervisorctl start` if it was
not.

Unfortunately, `list_supervisor_processes("zulip-tornado:*")` did not
include `zulip-tornado`, and as such a non-sharded process was always
considered to _not_ be running, and was thus started, not restarted.
Starting an already-started service is a no-op, and thus non-sharded
tornado processes were never restarted.

The observed behaviour is that requests to the tornado process attempt
to load the user from the cache, with a different prefix from Django,
and immediately invalidate the session and eject the user back to the
login page.

Fix the `list_supervisor_processes` logic to match without the
trailing `:*`.
2022-03-31 10:41:41 -07:00
Heidi Ahlberg
9e6836d0af i18n: Fix missing translation tags in stream creation view. 2022-03-31 10:33:34 -07:00
Greg Price
3ff2bcf62a shared: Bump version to 0.0.10. shared-0.0.10 2022-03-30 21:06:37 -07:00
Anders Kaseorg
7de1e7c477 changelog: Remove broken link.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-03-30 20:37:31 -07:00
Steve Howell
b7f670f5a0 markdown: Extract set_linkifier_regexes.
This is definitely better than having linkifiers
reach directly into marked.js, but there is
probably further improvement we can do here
to clean up how these regexes get set.

This introduces a circular dependency between
markdown.js and linkifiers.js, but we will
soon break it in the other direction.
2022-03-30 14:31:00 -07:00
Steve Howell
2a240d3e19 markdown: Move handleLinkifier.
All the other handleFoo functions followed this
convention.
2022-03-30 14:31:00 -07:00
Steve Howell
efdf2c8fe3 linkifiers: Avoid parallel data structure.
We can pretty easily work with a map in the
two places that we ever relied on an array.
2022-03-30 14:31:00 -07:00
Steve Howell
71c12e313c markdown: Fix overly loose regex for previews.
Fortunately, the only impact of this bug was that
we would unnecessarily wait for the server to render
the markdown if we got false matches.
2022-03-30 14:31:00 -07:00