Commit Graph

117 Commits

Author SHA1 Message Date
umkay
d260a22637 Add a new statistics/analytics framework.
This is a first pass at building a framework for collecting various
stats about realms, users, streams, etc. Includes:
* New analytics tables for storing counts data
* Raw SQL queries for pulling data from zerver/models.py tables
* Aggregation functions for aggregating hourly stats into daily stats, and
  aggregating user/stream level stats into realm level stats
* A management command for pulling the data

Note that counts.py was added to the linter exclude list due to errors
around %%s.
2016-10-04 17:18:54 -07:00
Tim Abbott
d25bbdf574 lint: Fix missing cast causing mypy errors. 2016-09-27 18:38:32 -07:00
Tim Abbott
aaa221233a lint: Ban 2-space indentation in our CSS.
Probably most properly we should check for any number of spaces that
isn't 4, but that's a bit more work to do with our linter framework,
and in practice basically every CSS whitespace error we see is 2-space.
2016-09-27 18:06:44 -07:00
Steve Howell
4c5eb3d06e Fix transaction behavior for update_subscriptions_backend().
This commit extracts compose_views() from update_subscriptions_backend(),
and it implements the correct behavior for forcing transactions to roll
back, which is to raise an exception.

There were really three steps in this commit:

- Extract buggy code to compose_views().
- Add tests on compose_views().
- Fix bugs exposed by the new tests by converting errors to exceptions.
2016-09-12 22:03:39 -07:00
Gordon P. Hemsley
c9d1a4247f Improve test coverage for tools.
* Replace generic Exception with TemplateParserException.
* Add tests to cover many of the uncovered lines in
  tools/lib/template_parser.py.
* Add an exclusion line to the naïve pattern for checking for missing
  tuples in format strings, to keep the linter happy.
2016-08-30 18:51:40 -04:00
Tim Abbott
2e2dee3396 lint: Ban lines longer than 140 characters without #ignorelongline. 2016-08-19 12:03:09 -07:00
Taranjeet Singh
7fc170d808 lint-all: Fix line with length greater than 120.
This updates file tools/lint-all.
2016-08-19 11:56:43 -07:00
Steve Howell
c6a888561a tools: Extract build_custom_checkers() in tools/lint-all. 2016-08-18 14:15:01 -07:00
Steve Howell
f955991120 tools: Add bright_red_output() to tools/lint-all 2016-08-18 14:15:01 -07:00
Steve Howell
ffe648baa8 tools: Extract EXCLUDED_FILES in tools/lint-all. 2016-08-18 14:15:01 -07:00
Steve Howell
dfe01c4f83 tools: Move run_parallel() out of run(). 2016-08-18 14:15:01 -07:00
Steve Howell
e3a7b9b1b2 tools: Move check_pyflakes() out of run(). 2016-08-18 14:15:01 -07:00
Steve Howell
3c40157195 tools: Add a run() method to tools/lint-all. 2016-08-18 14:15:01 -07:00
Christie Koehler
096b098410 Flush stdout and stderr prior to exiting child process.
This fixes a confusing bug that we ran into where the output from
certain child processes wouldn't appear when running `lint-all` via
`ssh` (without a tty) into the Vagrant environment.
2016-08-16 18:11:14 -07:00
Tim Abbott
c7059c9751 travis: Update success-http-headers to match current certs.
Travis CI seems to have changed the way the snakeoil SSL certs are
generated in their infrastructure, so we need to update our expected
"success" HTTP headers accordingly.
2016-08-12 09:35:41 -07:00
Steve Howell
a904222947 Add tools/find-add-class to tools/lint-all.
The find-add-class tool, when in lint mode, verifies that we can
understand all calls to addClass from our JS code.

When in non-lint mode, i.e. verbose mode, the tool prints out a
list of tuples of (fn, class) that we can use as we wish in other
tools.
2016-08-08 15:32:48 -07:00
Tim Abbott
0689485666 Add lint check for malformed type annotations missing :.
Also fix the 2 annotations that weren't being checked because of this.
2016-08-04 15:53:23 -07:00
Steve Howell
3b5c187f55 Add tools/check-css and tools/lib/css_parser.py.
`tools/lint-all` now calls the new `tools/check-css`

The css_parser library parsers CSS into a data structure
that remembers line numbers and columns of semantically
meaningful tokens and adjoining white space/tokens.  It
is intended to be used for various linting tasks.

The file `tools/check-css` runs a few files through the
parser and makes sure they round trip.  This has some value
right away, as files that fail to parse will cause an
exception to be thrown and thus alert developers to syntax
errors.  We expect to grow this into more advanced linting
tasks eventually.
2016-08-02 16:22:46 -07:00
Tim Abbott
75b5d021fa Remove unused puppet-common third-party module code. 2016-07-31 19:24:42 -07:00
Tim Abbott
568dd0e142 lint-all: Improve regular expression for json_error. 2016-07-28 16:19:55 -07:00
Steve Howell
713797a65c Add lint checks for self.client.{get,patch,put,post,delete}. 2016-07-27 20:50:54 -07:00
Eklavya Sharma
bd0fa3e77b Annotate tools/lint-all. 2016-07-24 21:04:31 +05:30
Tim Abbott
2338421c6d lint: Add documentation lint check for JavaScript spelling. 2016-07-12 19:22:21 -07:00
Tim Abbott
f513a68ac9 lint-all: Check for missing whitespace before { in CSS.
Also fix the existing violations of this rule.
2016-07-10 17:29:36 -07:00
Tim Abbott
88368397aa lint-all: Add Python check for space after if. 2016-07-10 10:58:16 -07:00
Steve Howell
e35b84d438 Have lint-all pass along -m to check-templates. 2016-07-08 16:30:49 -07:00
acrefoot
7ff89dc137 Add markdown-specific whitespace lint rule to /tools/lint-all.
Match one space or three+ spaces after content, or positive
number of spaces on an empty line.
2016-06-27 18:43:20 -07:00
Tim Abbott
7d3b451902 lint: Check for extremely long lines.
Currently, we check for lines longer than 180 characters; we can lower
this as we clean up or wrap longer strings.
2016-06-25 10:52:03 -07:00
Umair Khan
a70d5041d2 Don't allow literal strings in report_error. 2016-06-23 16:23:01 -07:00
Umair Khan
b7bb49c6f5 Don't allow literal string in report_success. 2016-06-23 16:23:01 -07:00
Tim Abbott
6b8e9c7254 Exclude static/locale from linters. 2016-06-21 17:17:34 -07:00
Tim Abbott
eb71173be3 lint: Fix warnings under confirmation/. 2016-06-20 08:19:54 -07:00
Tim Abbott
6d1d7471e6 Refactor out zproject/dev_settings.py. 2016-06-17 12:15:15 -07:00
Tim Abbott
d117ec8664 lint: Fix redundant stripping of strings. 2016-06-17 10:41:39 -07:00
Tim Abbott
267a71cf20 Run Python custom RE checks in parallel with non-python.
This saves about 1s (out of originally 3.3s) of time running all the
linters in my development environment.
2016-06-17 10:26:50 -07:00
John Hergenroeder
16a19226f6 Add linter check for redundant REQ whence argument. 2016-06-16 13:53:39 -07:00
Umair Khan
c1f5ac375c Add linter checks for javascript strings and HTML placeholders.
Checks are added for:
- i18n.t
- placeholder
- compose_error
2016-06-13 09:05:27 -07:00
Tim Abbott
584887e588 lint: Require folding of } on same line in else if statements. 2016-06-09 14:05:34 -07:00
Tim Abbott
c35781d505 lint: Require folding of } on same line as else statements. 2016-06-09 14:02:49 -07:00
Tim Abbott
0c1b5006f7 lint: Check for space after if in javascript. 2016-06-09 13:47:12 -07:00
Tim Abbott
eba0d6339f lint: Require space after // in JS comments. 2016-06-09 13:44:24 -07:00
Tim Abbott
a09b950097 lint-all: Check for relative URLs in JSON calls from JS. 2016-06-09 11:59:40 -07:00
Ernesto Vargas
ffb2f9e84b lint/tests: Give nice error message for common import failures.
This should make users much more likely to be able to debug issues
where they ran Zulip outside the Vagrant environment or virtualenv.

[error messages tweaked by tabbott]
2016-06-06 13:39:26 -07:00
Tim Abbott
553ef81f92 Add lint rules for mypy type annotations. 2016-06-05 13:00:55 -07:00
Tim Abbott
03debdf82f Fix malformed error message when creating invalid Realm Emoji.
Thanks to Greg McCoy for his help finding this bug.
2016-06-03 23:12:36 -07:00
Nathan Florea
5fe9076631 Remove some mutable default arguments.
These ones don't fix any bugs, because the mutable arg is never passed
outside of the callable or mutated.  But it's good practice to not use
them in case those invariants are changed in the future.
2016-06-03 09:16:56 -07:00
Umair Khan
bd4e471706 Check json_error and JsonableError in linter.
Mainly the check is to ensure that all the strings that pass through
these two functions are captured by `makemessages`.
2016-05-31 07:40:42 -07:00
Umair Khan
4b28fcd2f3 Add option to linter to exclude lines from files. 2016-05-31 07:40:42 -07:00
Tim Abbott
41336f3782 lint-all: Check for use of '== None'. 2016-05-31 07:02:04 -07:00
Eklavya Sharma
1c04560def Re-enable pyflakes in linter and remove python 3 pyflakes errors. 2016-05-25 19:25:13 +05:30