Commit Graph

320 Commits

Author SHA1 Message Date
Vishnu Ks
120ac21515 bugdown: Create a constant for inline preview limit. 2018-02-28 12:53:36 -08:00
Eeshan Garg
96036f07a3 api_code_example: Use json instead of ujson; specify separators.
The Markdown extension that lives inside
zerver/lib/bugdown/api_code_example.py previously used ujson.
ujson's `dumps` function doesn't accept a `separators` argument,
which means we have no control over how the JSON is pretty-printed.
This resulted in JSON fixtures with no spaces after the colon, which
looks unnecessarily convoluted.

So now, we use the built-in `json` module to get around this.

For further reading, this issue
<https://github.com/esnme/ultrajson/issues/82> opened on ujson's
repo explains why they are reluctant to support such formatting
due to performance considerations.
2018-02-20 20:10:38 -03:30
Steve Howell
46a49777c4 Add stream ids to urls for stream-related narrows.
This commit prefixes stream names in urls with stream ids,
so that the urls don't break when we rename streams.

strean name: foo bar.com%
before: #narrow/stream/foo.20bar.2Ecom.25
after: #narrow/stream/20-foo-bar.2Ecom.25

For new realms, everything is simple under the new scheme, since
we just parse out the stream id every time to figure out where
to narrow.

For old realms, any old URLs will still work under the new scheme,
assuming the stream hasn't been renamed (and of course old urls
wouldn't have survived stream renaming in the first place).  The one
exception is the hopefully rare case of a stream name starting with
something like "99-" and colliding with another stream whose id is 99.

The way that we enocde the stream name portion of the URL is kind
of unimportant now, since we really only look at the stream id, but
we still want a safe encoding of the name that is mostly human
readable, so we now convert spaces to dashes in the stream name.  Also,
we try to ensure more code on both sides (frontend and backend) calls
common functions to do the encoding.

Fixes #4713
2018-02-19 09:03:11 -08:00
Shubham Dhama
a32e1eb913 markdown: Require double-asterisk around all mentions.
This enforces `**` around all the mentions including "at-all" and
"at-everyone" mentions. Hence this makes `@all` and `@everyone`
invalid mentions, resulting into proper syntax for these mentions as
`@**all**` and `@**everyone**` respectively.

Note from tabbott: This removes an old feature/syntax, which made
sense back when @Tim was also a way to mention a user with Tim as
their first name.  Given how nice typeahead is now, the user part of
the feature was removed a while ago; this should have gone at the same
time.

Fixes: #8143.
2018-02-16 11:45:08 -08:00
Eeshan Garg
6206647641 bugdown/api_code_examples: Add support for multiple languages.
This commit modifies the Markdown extension in bugdown/api_code_examples.py
to support rendering code examples in multiple languages by specifying
the language like so:

{generate_code_example(python)|doc.md|example}

This makes us one step closer towards adding support for testable
JavaScript code examples.
2018-02-16 10:07:13 -08:00
Tim Abbott
77addc5456 bugdown: Fix handling of ultra-long renderings.
If some bug in Bugdown results in a rendered message content that is
bigger than twice the message size, we now just throw an exception
from Bugdown.  This is considerably better than the old behavior,
which might result in an enormous message being placed in the database
(potentially, bigger than the 1MB limit to store in memcached), which
would in turn result in tragic consequences.

This fixes #8322, in that it prevents the super bad outcome seen there
(where basically Zulip became unusable for everyone on the stream
where the message is posted).  Now, the failure mode is just the
message failing to send.  Still not ideal (and requires further work
on the URL embed feature), but not a minor problem, not a major one.
2018-02-09 10:57:55 -08:00
Eeshan Garg
09721e9ec3 bugdown/api_code_examples: Support multiple fixtures per doc.
This commit adds support for passing in an argument to the macro
"call" to explicitly specify a fixture to render, like so:

{generate_code_example|doc_name|fixture(stream_message_with_args)}
2018-02-07 15:58:57 -08:00
Eeshan Garg
0cca39d1fc bugdown/api_code_examples: Support multiple examples in one test.
This commit adds support for recursively extracting multiple code
examples from a single test method.
2018-02-07 15:58:57 -08:00
Eeshan Garg
ffe930d50d bugdown/api_generate_examples: Replace 'method' with 'example'.
To generate a code exammple,

{generate_code_example|<api_doc_md>|example} sounds better and
more intuitive than,

{generate_code_example|<api_doc_md>|method}
2018-02-07 15:58:57 -08:00
rht
9161f8c39b zerver/lib: Remove u prefix from strings. 2018-02-05 12:12:58 -08:00
Eeshan Garg
fe98a59f5c bugdown/api_code_examples: Add macro for admin zuliprc lines.
Some of our code examples can only be run with administrator
credentials (such as create-user). Thus, the Markdown extension
for generating code examples should have an option to include
the lines that recommend using an admin zuliprc instead of a
non-admin one.
2018-01-31 07:30:54 -05:00
Eeshan Garg
c158869096 Rename tools/lib/api_tests.py to zerver/lib/api_test_helpers.py.
Now that the Markdown extension defined in
zerver/lib/bugdown/api_generate_examples depended on code in the
tools/lib/* directory, it caused the production tests to fail since
the tools/ directory wouldn't exist in a production environment.
2018-01-31 07:30:54 -05:00
Eeshan Garg
c89550cfa0 api docs: Add Markdown extension for generating examples/fixtures.
This commit adds a Markdown extension that allows the following
syntax,

{generate_code_example|<md_file_name>|<fixture or method>}

to generate code examples and fixtures found in tools/lib/api_tests.py
and templates/zerver/api/fixtures.json, respectively.
2018-01-31 07:30:54 -05:00
rht
ee06da9350 bugdown: Linkify Bitcoin URI. 2018-01-19 06:07:43 -05:00
Shreyansh Dwivedi
b0fb7aa6b2 bugdown: Add preview for vimeo videos.
This also amends a commit from Brock Whittaker <brock@zulipchat.com>
that merges two separate functions for YouTube videos and Vimeo videos
into a generic video recall function.

Fixes #7550.
2018-01-17 09:11:43 -05:00
Andy Perez
c209069fcd bugdown: Improve image inlining logic.
Fix #7537
2018-01-09 16:43:22 -05:00
Eeshan Garg
89f622937d api docs: Generate Arguments table from JSON.
This commit does the following:
* Move the Arguments table data from stream-message.md and
  private-message.md to a JSON file.
* Add a Markdown extension that allows one to include and render
  a table from a JSON file like so:
    {generate_arguments_table|arguments.json|private-stream.md}
* Use Bootstrap's .table class to format the table instead of
  relying on custom CSS.
2017-12-29 17:03:36 -05:00
akashnimare
ef51a39552 markdown: Update regex for strikethrough.
This uses the correct regex for strikethrough. Also, added
a test to make sure that strikethrough works when it contains
link with whitespace.

Fixes #7596.
2017-12-28 15:59:04 -05:00
Greg Price
b15231dfc2 logging: Rename AdminZulipHandler to AdminNotifyHandler.
This name hasn't been right since f7f2ec0ac back in 2013; this handler
sends the log record to a queue, whose consumer will not only maybe
send a Zulip message but definitely send an email.  I found this
pretty confusing when I first worked on this logging code and was
looking for how exception emails got sent; so now that I see exactly
what's actually happening here, fix it.
2017-12-22 11:52:13 -05:00
Tommy Ip
9ea7b05cfe bugdown: Rewrite markdown link to relative link conditionally. 2017-12-22 08:10:42 -05:00
Tommy Ip
efe2d497f4 bugdown: Avoid matching uri with regex. 2017-12-22 08:10:42 -05:00
Tommy Ip
9ce797d68c bugdown: Extract rewrite_if_relative_link function. 2017-12-22 08:10:42 -05:00
Andy Perez
f0190333b8 bugdown: Remove whitespace strip from inline code.
Fixes #4507
2017-12-14 07:48:10 -05:00
Weronika Grzybowska
b54f78968f markdown: Add automatic numbering of ordered lists.
Adds a markdown preprocessor that finds ordered lists where all items
use the same number and change them to be in normal increasing order,
starting with that number.

Fixes #5159.
2017-12-12 18:35:38 -06:00
Xavier Cooney
a831acd97f mypy: Use Python 3 type syntax in bugdown/__init__.py. 2017-12-08 18:34:26 -08:00
rht
ee546a33a3 zerver/lib: Use python 3 syntax for typing.
Edited by tabbott to improve various line-wrapping decisions.
2017-11-28 17:15:14 -08:00
Rohitt Vashishtha
6734b00c0b markdown: Hide URL if message is only an image.
Hides URL if the message content == image url so that sending gifs or
images feels less cluttered. Uses the url_to_a() function to generate
the expected url string for matching.

Fixes #7324.
2017-11-27 13:30:18 -08:00
Andy Perez
0449dc8b8a bugdown: Fix handling of nested fenced math blocks.
Change any fence delimiters (backticks and tildes) to be
interpreted literally inside of math fenced_code blocks.

Fixes #5190.
2017-11-22 12:19:43 -08:00
rht
3f4bf2d22f zerver/lib: Use python 3 syntax for typing.
Extracted from a larger commit by tabbott because these changes will
not create significant merge conflicts.
2017-11-21 20:56:40 -08:00
Andy Perez
83f563aeff bugdown: Properly parse empty blockquotes.
The character ">" now only starts a blockquote if the resulting
blockquote would be non-empty.  Thus, by itself, ">" is now
interpreted literally by bugdown, fixing #687.  The message
with contents consisting of ">>>" is now parsed as a doubly
(not triply) nested blockquote with contents ">".  Properly
formed blockquotes have identical behavior as before, but now
bugdown can no longer produce empty blockquotes as output.

Fixes #2886, #687.
2017-11-17 11:41:44 -08:00
David Rosa Tamsen
7072fa5b37 docs: Reorganize developer docs to improve navigation.
This commit helps reduce clutter on the navigation sidebar.
Creates new directories and moves relevant files into them.
Modifies index.rst, symlinks, and image paths accordingly.

This commit also enables expandable/collapsible navigation items,
renames files in docs/development and docs/production,
modifies /tools/test-documentation so that it overrides a theme setting,
Also updates links to other docs, file paths in the codebase that point
to developer documents, and files that should be excluded from lint tests.

Note that this commit does not update direct links to
zulip.readthedocs.io in the codebase; those will be resolved in an
upcoming follow-up commit (it'll be easier to verify all the links
once this is merged and ReadTheDocs is updated).

Fixes #5265.
2017-11-16 09:45:08 -08:00
Harshit Bansal
3b95a6d2d9 bugdown: Fix a bug in the updation logic of markdown engines.
While fixing an issue related to email gateway messages not getting
rendered properly, I unknowingly introduced a bug in the markdown
engine updation code. This commit fixes it. The issue was that for
a realm having email gateway setup, updation of realm filters would
lead to the updation of only one of the markdown engines not both.
2017-11-15 22:47:11 -08:00
Harshit Bansal
b849c5e551 bugdown: Remove an unnecessary check in do_convert(). 2017-11-15 22:47:11 -08:00
Harshit Bansal
5e5bb02e39 bugdown: Remove make_realm_filters().
This commit removes `make_realm_filters()` and merges its functionality
in `make_md_engine()` and `maybe_update_markdown_engines()`.
2017-11-15 22:47:11 -08:00
Harshit Bansal
f9f9e92443 bugdown: Correct the descriptions of Bugdown config. 2017-11-15 22:47:11 -08:00
Harshit Bansal
e615a2ab4f bugdown: Simplify the "opts" parameter of make_md_engine(). 2017-11-15 22:47:11 -08:00
Harshit Bansal
e3f2f67d0e bugdown: Rename maybe_update_realm_filters().
This commit renames `maybe_update_realm_filters()` to
`maybe_update_markdown_engines()` which sounds closer
to the semantics.
2017-11-15 22:47:11 -08:00
Andy Perez
4d51a6cfac bugdown: Fix $$ whitespace handling.
The intended use of $$ is for inline expressions, not for multiline
ones; ```math is an acceptable alternative for the latter.  Hence,
the $$-syntax for inline TeX no longer permits newlines within it.

This was also necessary for the next change to be sensible; namely
allowing for spaces around both $$ when crafting inline TeX instead of
forcing everything to be crammed together, e.g. $$x=7$$.  In order to
avoid uninentionally creating inline expressions, the opening and
closing $$'s of an inline expression must now both exactly consist of
two dollar signs, no more and no less.

Fixes: #6488.
2017-11-10 16:54:27 -08:00
rht
8e87eb4084 zerver/lib/bugdown: Text-wrap long lines exceeding 110. 2017-11-10 16:29:32 -08:00
Umair Khan
18aecb82d7 user-groups: Get members from membership. 2017-11-09 16:18:40 -08:00
Tommy Ip
bcd575d8de bugdown: Rewrite links to local server to be relative links.
Fixes #7247
2017-11-09 15:59:31 -08:00
Steve Howell
531a3ee9de refactor: Fix parse.quote(force_str(s)) calls in bugdown. 2017-11-09 10:43:20 -08:00
Steve Howell
17cfead19a Remove force_text() in twitter_text(). 2017-11-09 10:43:20 -08:00
Steve Howell
585829f665 Remove force_text() in list_of_tlds(). 2017-11-09 10:43:19 -08:00
Steve Howell
786e90d774 Removed unused imports of force_bytes. 2017-11-09 10:32:14 -08:00
Steve Howell
a77cdaa527 Remove force_text() in get_url_data(). 2017-11-09 10:32:14 -08:00
Harshit Bansal
65838bb825 email_gateway: Disable code block processor for email gateway.
Generally emails are not written with markdown in mind and hence
sometimes render in strange ways. This commit fixes a particular
issue that was causing whitespace before paragraphs to be treated
as code block due to which email content was being rendered in a
box that scrolls in right direction a lot.

Fixes: #7045.
2017-11-09 09:56:35 -08:00
derAnfaenger
19bc55aa45 Fix various typos.
The typos and their corrections were found with the
aid of https://github.com/lucasdemarchi/codespell.
2017-11-09 16:26:38 +01:00
rht
8242c15b48 zerver/lib: Remove unused imports (F401). 2017-11-07 16:37:13 -08:00
rht
01eb22abd9 refactor: Remove all the remaining six.moves import. 2017-11-07 10:51:44 -08:00