Commit Graph

2519 Commits

Author SHA1 Message Date
Umair Khan
68513952fb email-worker: Create EmailSendingWorker.
This commit just copies all the code from MissedMessageSendingWorker
class to a new EmailSendingWorker class. All the logic to send an email
through a queue was already there. This commit only makes the logic
generic. It does so by creating a special purpose queue called
'email_senders' to send any type of email. To make
MissedMessageSendingWorker still work we derive it from
EmailSendingWorker. All the tests that were testing
MissedMessageSendingWorker now run against EmailSendingWorker.
2017-12-20 19:36:27 -08:00
Vishnu Ks
f743eeb1eb circleci: Set up locale en_US.UTF-8.
See comment for motivation and what's known.

[greg: Added comment, summarizing Vishnu's results from testing.]
2017-12-20 19:18:29 -08:00
Greg Price
eb650306eb circleci: Disable gitlint for now.
We get the following error (edited slightly):

  Dec 19 06:13:27 commit_messages| An error occurred while executing
    '/usr/bin/git rev-list --max-count=-1 upstream/master..HEAD':
  fatal: ambiguous argument 'upstream/master..HEAD':
    unknown revision or path not in the working tree.
  Use '--' to separate paths from revisions, like this:
  'git <command> [<revision>...] -- [<file>...]'

We'll need to adjust the remotes to make `upstream` mean what we expect.
2017-12-18 22:21:49 -08:00
Greg Price
e5c6c446b5 travis + circleci: Start postgresql during provision.
Otherwise, in CircleCI it isn't running when we need it later.
Travis must special-case starting it automatically.
2017-12-18 22:05:27 -08:00
Greg Price
c83c118c6c circleci: Crudely clone a bit of is_travis logic.
This isn't really the right way to do this -- commit
  dcd80e665 "travis/setup-backend: Remove the '--travis' flag"
took us in the wrong direction by introducing more magic, deeper in
the stack.

But it's the same way we do it for Travis.  For now, just copy that.

[Thanks to hackerkid for cleaning up my original crude hack.]
2017-12-18 22:04:39 -08:00
Greg Price
e8bf7d23bf provision: Remove transitional Python 2->3 logic in setup_venvs.
At this point if we were accidentally using `/srv/zulip-venv` for
anything, we'd have run into it by now.  So just drop the bit of
historical logic that we had to ensure that.
2017-12-18 21:59:51 -08:00
Greg Price
3ba0d5659f travis: Don't create nagios state.
As the comment says, we don't seem to use this.  And the Travis builds
work fine without it.
2017-12-18 21:43:07 -08:00
cPhost
3dc315a0a1 node tests: Add 100% coverage for narrow_state.js. 2017-12-18 09:57:43 -05:00
Tommy Ip
7243a91e47 linter: Re-enable gitlint. 2017-12-18 09:36:47 -05:00
Greg Price
137c0e65bb tools: Revert to Python 2 typing syntax for now.
This reverts commit 66261f1cc.  See parent commit for reason; here,
provision worked but `tools/run-dev.py` would give errors.

We need to figure out a test that reproduces these issues, then make a
version of these changes that keeps that test working, before we
re-merge them.
2017-12-13 10:38:15 -08:00
Greg Price
d5fd2a1d72 tools: Fix run-dev.py, which had an if False: on a needed import. 2017-12-12 20:50:00 -08:00
rht
66261f1cc3 tools: Use Python 3 syntax for typing in many files. 2017-12-12 17:42:57 -08:00
Symt
3706ec7479 linter: Add good/bad lines for CSS and HTML. 2017-12-11 15:37:22 -06:00
Steve Howell
738c73f08e lint: Prevent self: Any annotations.
We should omit these for mypy.  For most class definitions,
mypy doesn't need `Any`, and it provides no real useful info.

For clever monkeypatches, you should provide a more specific
type than `Any`.
2017-12-08 19:00:18 -08:00
cPhost
c6985e2eee eslint: add --cache flag for faster linting 2017-12-07 15:14:51 -08:00
picapi_
96fa399ce0 mypy: Use Python 3 type syntax in tools/lister.py. 2017-12-05 16:32:18 -08:00
Greg Price
ff435439af circle-docker: Describe how to build the image.
These commands are super boring standard Docker commands,
so this probably isn't helpful for anyone who is familiar
with building Docker images... but I had to consult docs
to work out the right commands again today, so they'd help me.
2017-12-01 17:16:51 -08:00
Greg Price
535fd9bb5c circle-docker: Silence some spam output. 2017-12-01 17:16:51 -08:00
Greg Price
fda349fd33 circle-docker: Fix getting a sudo prompt.
Provision was failing at tools/setup/postgres-init-dev-db
with this in the log:

    Sep 20 02:27:01 + sudo -i -u postgres psql ''
    [sudo] password for circleci:

The issue is that the old version of this line (from Circle upstream)
only lets the `circleci` user sudo to root -- not to other users, or
not directly anyway -- because sudoers syntax is complicated.  Fix it,
after studying `man sudoers`.
2017-12-01 15:21:28 -08:00
Greg Price
a1265e09ba circle-docker: Add all the standard APT deps we provision.
This is only an optimization -- if this list is missing anything,
we'll get to it in `provision` in the actual build.  That's important,
because we want an existing image to work fine for testing new
versions of our codebase, including changes that may install more
packages in `provision`.

What this does accomplish is keeping provision's `apt-get install`
fast, by leaving it very little work to do.

The list comes from looking at the APT output during provision in an
actual run without this step, and leaving out two packages which
aren't available at this stage, because we get them from PPAs:
postgresql-9.3-pgroonga and postgresql-9.3-tsearch-extras.
2017-12-01 15:21:28 -08:00
Greg Price
ab57756d61 circle-docker: Prune redundant packages from install list for clarity. 2017-12-01 15:21:28 -08:00
Greg Price
642fb8537e circle-docker: Use -scm variant of base image to save a bit of work. 2017-12-01 15:21:28 -08:00
Greg Price
8050ad6c3b circle-docker: Simplify a bit for clarity and efficiency.
Install `jq` with APT -- that's a lot simpler to read than this
explicit download.

And coalesce several commands, following Docker upstream's
recommendation and avoiding unnecessary overhead.
2017-12-01 15:21:28 -08:00
Greg Price
7094f78302 circle: First, near-upstream cut at a Dockerfile.
This is nearly the same as Circle's version, linked in the comment.
I've
* changed the FROM line to get Ubuntu,
* added a couple of distro packages to compensate, and
* revised the comments.
2017-12-01 14:47:55 -08:00
Tim Abbott
e9b3ac3f34 travis: Remove now-unnecssary items from apt-mark hold lists.
This seems to have been causing the travis production suite to fail.
It's a direct consequence of removing travis' giant library of apt
sources.list files; now that those are gone, there aren't copies of
all these extra packages available anyway.
2017-11-30 14:56:36 -08:00
Tim Abbott
6d887dc6ee test-backend: Require 100% test coverage on confirmation code. 2017-11-29 22:20:06 -08:00
Greg Price
c32b16715d tornado: Use spiffy new call_later rather than add_timeout.
This method was new in Tornado 4.0.  It saves us from having to get
the time ourselves and do the arithmetic -- which not only makes the
code a bit shorter, but also easier to get right.  Tornado docs (see
http://www.tornadoweb.org/en/stable/ioloop.html) say we should have
been getting the time from `ioloop.time()` rather than hardcoding
`time.time()`, because the loop could e.g. be running on the
`time.monotonic()` clock.
2017-11-29 16:56:29 -08:00
Vishnu Ks
5279ac4601 droplets: Update snapshot id. 2017-11-29 14:24:08 -08:00
Eeshan Garg
bae1ecf1c5 tools/setup: Delete static/generated/bots before repopulating.
tools/setup/generate_zulip_bots_static_files now starts off by
deleting static/generated/bots/ (if it hasn't been removed already)
so that outdated static files from older versions of the zulip_bots
package don't supress errors in the main repo that would otherwise
break.

For more info, see #7542

Fixes: #7542.
2017-11-28 17:17:54 -08:00
Greg Price
f632692188 lint: Fix use of re.match in judging long lines.
The `re.match` function in the Python stdlib is a trap for the unwary,
with surprising and asymmetrical semantics; we should probably add a
lint rule to ban it entirely.  The docstring says:

> Try to apply the pattern at the start of the string, [...]

In other words, it effectively adds a `^` at the start (or `\A`, where
the distinction matters.)  It's bad enough that this differs from what
grep, sed, perl, less, and every other tool I can think of do when
looking for matches to a regex; on top of that, it treats the
beginning of the string differently from the end, for no obvious
reason.  The function that does what the rest of the world understands
by "match against this regex" is `re.search`.

In this case, it's unlikely that anyone intended for comments with
URLs, or `api_url` references, to miss out on their respective
exceptions to the long-line rule if they happen to start after the
first column.  So fix those rules by just switching to `re.search`
with the same pattern.

I think Markdown URL references may have to start at the beginning of
the line, so I've left a `^` there to preserve -- but now make
explicit -- the `re.match` behavior.
2017-11-28 16:23:10 -08:00
Umair Khan
274bba82b9 two_factor: Add configuration and URLs.
This adds django-two-factor to the project, but held behind
settings.TWO_FACTOR_AUTHENTICATION_ENABLED, so that this has no effect
by default.
2017-11-28 15:21:40 -08:00
Tommy Ip
2d7536bf05 Document the add_mentor.py tool. 2017-11-28 13:26:59 -08:00
Tommy Ip
ba1a32ca74 Create script to add and remove mentor's ssh key from DO droplets. 2017-11-28 13:26:59 -08:00
Vishnu Ks
87a412a992 droplets: Fix broken link for requesting droplets. 2017-11-28 08:53:52 -08:00
Vishnu Ks
73ee9a4ad8 droplets: Update info on recreating the droplets. 2017-11-28 08:53:52 -08:00
Vishnu Ks
c37193568f droplets: Update baseimage snapshot id.
The new snapshot has python-zulip-env as well
as VIM and mosh installed.
2017-11-28 08:53:52 -08:00
Tommy Ip
19b518c801 refactor: Extract tests for upload mechanics.
This temporarily removes the tests for clear_out_file_list since
fixing that test proved to be difficult.
2017-11-27 21:34:55 -08:00
Tommy Ip
ddaff4cd2a refactor: Extract upload mechanics to new JS module.
Tweaked by tabbott to move changes from the next commit that are
required for this to pass tests into this commit.

Note that this exports a few items that were not previously exported.
2017-11-27 21:31:51 -08:00
Eeshan Garg
880fd5497a markdown: Add indented versions of multi-line Markdown macros.
This is part of our efforts to change our integrations/webhooks
docs to follow the same sort of numbered-list format as our /help
docs. In order to indicate that paragraphs separated by newlines
are part of the same numbered-list point, every paragraph must be
indented 4 spaces.
2017-11-25 17:01:11 -08:00
Rhea Parekh
84847b811b requirements: Upgrade mypy to 0.550.
We have to add a few type: ignores due to mypy 0.550 having some bugs
in the new imaplib stubs in typeshed.

Fixes #7387.
2017-11-25 10:06:27 -08:00
Tim Abbott
d639e31641 lint: Fix check-templates handling of team.html.
This prevents some unnecessary error messages when running the linter
on a single file.
2017-11-25 09:43:01 -08:00
Tim Abbott
6f53d65e56 docs: Update a few broken links to the Git guide. 2017-11-23 16:24:00 -08:00
Tim Abbott
4d4532a27e copy_and_paste: Add a simple extensible test suite.
This commit is mostly about adding the test suite; some of the results
are things we'd like to change.
2017-11-23 12:59:01 -08:00
Tim Abbott
15501c8b07 lint: Clean up json_rules logic for tab-based whitespace.
This both improves the comment to be more readable, and also uses the
new and improved exclude feature to limit the exclusion to just the
webhook fixtures (where it's needed).

Also fixes a mypy error.
2017-11-23 12:01:20 -08:00
Tim Abbott
fb88eaa2d8 lint: Fix logic for exclude to handle directory trees.
The previous exclude rules only allowed excluding a directory (and
things in subdirectories would silently still be linted).  Anyone
using this would expect it to exclude a directory tree, so we make it
do that.
2017-11-23 12:01:20 -08:00
Vishnu Ks
def573cde3 remotedev: Add option to recreate droplets. 2017-11-23 11:09:30 -08:00
Tim Abbott
938892db8f lint: Ban tab-based whitespace in our our text fixtures.
We don't want to do this for the webhook fixtures since those come
from third-party code, but it makes sense to do this for hand-edited
code.
2017-11-23 10:47:09 -08:00
Tim Abbott
a4e9da22e0 coverage: Update pattern for __str__ for new typing syntax.
The new Python 3 typing syntax means the def lines for __str__ can
look a bit different.
2017-11-22 12:16:58 -08:00
Tommy Ip
e44b8809b2 /team: Render contributors list in tabs client side. 2017-11-22 11:33:24 -08:00
Tommy Ip
09d103091d /team: Use list instead of dict for contributors data. 2017-11-22 11:33:24 -08:00