mirror of
https://github.com/zulip/zulip.git
synced 2025-11-04 05:53:43 +00:00
Update testing notes formatting and content.
(imported from commit 44cc34888f14d101c29bcd1c8d63cc082b4ffe09)
This commit is contained in:
Yoyo Zhou
190
docs/testing.rst
190
docs/testing.rst
@@ -8,16 +8,20 @@ Postgres testing databases
|
||||
``tools/postgres-init-test-db`` works like ``tools/postgres-init-db`` to
|
||||
create the database "zulip\_test" and set up the necessary permissions.
|
||||
|
||||
.. attention::
|
||||
TODO: Is the below still accurate?
|
||||
|
||||
``tools/do-destroy-rebuild-test-database`` is an alias for
|
||||
``zephyr/tests/generate-fixtures --force``. Do this after creating the
|
||||
``tools/generate-fixtures --force``. Do this after creating the
|
||||
postgres database.
|
||||
|
||||
NB: Running ``generate-fixtures`` attempts to restore "zulip\_test" from
|
||||
a template database that was created. It's an unsafe copy (assumes that
|
||||
no one is writing to the template db, and that the db being copied into
|
||||
is DROP-able), so don't try to run the dev server and run tests at the
|
||||
same time, though ``generate-fixtures --force`` should make things happy
|
||||
again.
|
||||
.. note::
|
||||
Running ``generate-fixtures`` attempts to restore "zulip\_test" from
|
||||
a template database that was created. It's an unsafe copy (assumes that
|
||||
no one is writing to the template db, and that the db being copied into
|
||||
is DROP-able), so don't try to run the dev server and run tests at the
|
||||
same time, though ``generate-fixtures --force`` should make things happy
|
||||
again.
|
||||
|
||||
Wiping the test databases
|
||||
-------------------------
|
||||
@@ -34,6 +38,14 @@ If that fails you should try to do:
|
||||
|
||||
and then run ``tools/do-destroy-rebuild-test-database``
|
||||
|
||||
Recreating the postgres cluster
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. warning::
|
||||
|
||||
**This is irreversible, so do it with care, and never do this anywhere
|
||||
in production.**
|
||||
|
||||
If your postgres cluster (collection of databases) gets totally trashed
|
||||
permissions-wise, and you can't otherwise repair it, you can recreate
|
||||
it. On Ubuntu:
|
||||
@@ -43,9 +55,6 @@ it. On Ubuntu:
|
||||
sudo pg_dropcluster --stop 9.1 main
|
||||
sudo pg_createcluster --locale=en_US.utf8 --start 9.1 main
|
||||
|
||||
**This is irreversible, so do it with care, and never do this anywhere
|
||||
except your personal work machine (desktop or laptop).**
|
||||
|
||||
Running tests
|
||||
=============
|
||||
|
||||
@@ -60,7 +69,7 @@ Schema and initial data changes
|
||||
|
||||
If you change the database schema or change the initial test data, you
|
||||
have have to regenerate the pristine test database by running
|
||||
``zephyr/tests/generate-fixtures --force``.
|
||||
``tools/generate-fixtures --force``.
|
||||
|
||||
Writing tests
|
||||
=============
|
||||
@@ -70,12 +79,13 @@ We have a list of `test cases to write <Test%20cases%20to%20write>`__.
|
||||
Backend Django tests
|
||||
--------------------
|
||||
|
||||
These live in ``zephyr/tests.py``.
|
||||
These live in ``zerver/tests.py`` and ``zerver/test_*.py``. Run them
|
||||
with ``tools/test-backend``.
|
||||
|
||||
Web frontend black-box tests
|
||||
----------------------------
|
||||
|
||||
These live in ``zephyr/tests/frontend/tests.js``. This is a "black box"
|
||||
These live in ``zerver/tests/frontend/tests/``. This is a "black box"
|
||||
test; we load the frontend in a real (headless) browser, from a real dev
|
||||
server, and simulate UI interactions like sending messages, narrowing,
|
||||
etc.
|
||||
@@ -83,7 +93,7 @@ etc.
|
||||
Since this is interacting with a real dev server, it can catch backend
|
||||
bugs as well.
|
||||
|
||||
You can run this with ``./zephyr/tests/frontend/run``. You will need
|
||||
You can run this with ``./zerver/tests/frontend/run``. You will need
|
||||
`PhantomJS <http://phantomjs.org/>`__ 1.7.0 or later.
|
||||
|
||||
Debugging Casper.JS
|
||||
@@ -94,7 +104,7 @@ is not perfect. Here are some steps for using it and gotchas you might
|
||||
want to know.
|
||||
|
||||
To turn on remote debugging, pass ``--remote-debug`` to the
|
||||
``./zephyr/frontend/tests/run`` script. This will run the tests with
|
||||
``./zerver/frontend/tests/run`` script. This will run the tests with
|
||||
port ``7777`` open for remote debugging. You can now connect to
|
||||
``localhost:7777`` in a Webkit browser. Somewhat recent versions of
|
||||
Chrome or Safari might be required.
|
||||
@@ -121,9 +131,11 @@ individual JavaScript files that use the module pattern. For example, to
|
||||
test the ``foobar.js`` file, you would first add the following to the
|
||||
bottom of ``foobar.js``:
|
||||
|
||||
| `` if (typeof module !== 'undefined') {``
|
||||
| `` module.exports = foobar;``
|
||||
| `` }``
|
||||
::
|
||||
|
||||
if (typeof module !== 'undefined') {
|
||||
module.exports = foobar;
|
||||
}
|
||||
|
||||
This makes ``foobar.js`` follow the CommonJS module pattern, so it can
|
||||
be required in Node.js, which runs our tests.
|
||||
@@ -132,36 +144,44 @@ Now create ``zerver/tests/frontend/node/foobar.js``. At the top, require
|
||||
the `Node.js assert module <http://nodejs.org/api/assert.html>`__, and
|
||||
the module you're testing, like so:
|
||||
|
||||
| `` var assert = require('assert');``
|
||||
| `` var foobar = require('js/foobar.js');``
|
||||
::
|
||||
|
||||
var assert = require('assert');
|
||||
var foobar = require('js/foobar.js');
|
||||
|
||||
(If the module you're testing depends on other modules, or modifies
|
||||
global state, you need to also read the next section.)
|
||||
global state, you need to also read `the next section`__.)
|
||||
|
||||
__ handling-dependencies_
|
||||
|
||||
Define and call some tests using the `assert
|
||||
module <http://nodejs.org/api/assert.html>`__. Note that for "equal"
|
||||
asserts, the *actual* value comes first, the *expected* value second.
|
||||
|
||||
| `` (function test_somefeature() {``
|
||||
| `` assert.strictEqual(foobar.somefeature('baz'), 'quux');``
|
||||
| `` assert.throws(foobar.somefeature('Invalid Input'));``
|
||||
| `` }());``
|
||||
::
|
||||
|
||||
(function test_somefeature() {
|
||||
assert.strictEqual(foobar.somefeature('baz'), 'quux');
|
||||
assert.throws(foobar.somefeature('Invalid Input'));
|
||||
}());
|
||||
|
||||
The test runner (index.js) automatically runs all .js files in the
|
||||
zerver/tests/frontend/node directory.
|
||||
|
||||
.. _handling-dependencies:
|
||||
|
||||
Handling dependencies in tests
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
(From an email from Steve, August 21st.)
|
||||
|
||||
The following scheme helps avoid tests leaking globals between each
|
||||
other.
|
||||
|
||||
First, if you can avoid globals, do it, and the code that is directly
|
||||
under test can simply be handled like this:
|
||||
|
||||
`` var search = require('js/search_suggestion.js');``
|
||||
::
|
||||
|
||||
var search = require('js/search_suggestion.js');
|
||||
|
||||
For deeper dependencies, you want to categorize each module as follows:
|
||||
|
||||
@@ -173,26 +193,30 @@ For deeper dependencies, you want to categorize each module as follows:
|
||||
For all the modules where you want to run actual code, add a statement
|
||||
like the following to the top of your test file:
|
||||
|
||||
| `` add_dependencies({``
|
||||
| `` _: 'third/underscore/underscore.js',``
|
||||
| `` util: 'js/util.js',``
|
||||
| `` Dict: 'js/dict.js',``
|
||||
| `` Handlebars: 'handlebars',``
|
||||
| `` Filter: 'js/filter.js',``
|
||||
| `` typeahead_helper: 'js/typeahead_helper.js',``
|
||||
| `` stream_data: 'js/stream_data.js',``
|
||||
| `` narrow: 'js/narrow.js'``
|
||||
| `` });``
|
||||
::
|
||||
|
||||
add_dependencies({
|
||||
_: 'third/underscore/underscore.js',
|
||||
util: 'js/util.js',
|
||||
Dict: 'js/dict.js',
|
||||
Handlebars: 'handlebars',
|
||||
Filter: 'js/filter.js',
|
||||
typeahead_helper: 'js/typeahead_helper.js',
|
||||
stream_data: 'js/stream_data.js',
|
||||
narrow: 'js/narrow.js'
|
||||
});
|
||||
|
||||
For modules that you want to completely stub out, please use a pattern
|
||||
like this:
|
||||
|
||||
| `` set_global('page_params', {``
|
||||
| `` email: 'bob@zulip.com'``
|
||||
| `` });``
|
||||
::
|
||||
|
||||
| `` // then maybe further down``
|
||||
| `` global.page_params.email = 'alice@zulip.com';``
|
||||
set_global('page_params', {
|
||||
email: 'bob@zulip.com'
|
||||
});
|
||||
|
||||
// then maybe further down
|
||||
global.page_params.email = 'alice@zulip.com';
|
||||
|
||||
Finally, there's the hybrid situation, where you want to borrow some of
|
||||
a module's real functionality but stub out other pieces. Obviously, this
|
||||
@@ -200,18 +224,20 @@ is a pretty strong smell that the other module might be lacking in
|
||||
cohesion, but that code might be outside your jurisdiction. The pattern
|
||||
here is this:
|
||||
|
||||
| `` // Use real versions of parse/unparse``
|
||||
| `` var narrow = require('js/narrow.js');``
|
||||
| `` set_global('narrow', {``
|
||||
| `` parse: narrow.parse,``
|
||||
| `` unparse: narrow.unparse``
|
||||
| `` });``
|
||||
::
|
||||
|
||||
| `` // But later, I want to stub the stream without having to call super-expensive``
|
||||
| `` // real code like narrow.activate().``
|
||||
| `` global.narrow.stream = function () {``
|
||||
| `` return 'office';``
|
||||
| `` };``
|
||||
// Use real versions of parse/unparse
|
||||
var narrow = require('js/narrow.js');
|
||||
set_global('narrow', {
|
||||
parse: narrow.parse,
|
||||
unparse: narrow.unparse
|
||||
});
|
||||
|
||||
// But later, I want to stub the stream without having to call super-expensive
|
||||
// real code like narrow.activate().
|
||||
global.narrow.stream = function () {
|
||||
return 'office';
|
||||
};
|
||||
|
||||
Coverage reports
|
||||
~~~~~~~~~~~~~~~~
|
||||
@@ -219,11 +245,15 @@ Coverage reports
|
||||
You can automatically generate coverage reports for the JavaScript unit
|
||||
tests. To do so, install istanbul:
|
||||
|
||||
`` sudo npm install -g istanbul``
|
||||
::
|
||||
|
||||
sudo npm install -g istanbul
|
||||
|
||||
And run test-js-with-node with the 'cover' parameter:
|
||||
|
||||
`` tools/test-js-with-node cover``
|
||||
::
|
||||
|
||||
tools/test-js-with-node cover
|
||||
|
||||
Then open ``coverage/lcov-report/js/index.html`` in your browser.
|
||||
Modules we don't test *at all* aren't listed in the report, so this
|
||||
@@ -249,20 +279,21 @@ test messages involving Shakespeare characters.
|
||||
Testing signups
|
||||
---------------
|
||||
|
||||
The logic behind signups is dependant on the setting of
|
||||
The logic behind signups is dependent on the setting of
|
||||
``ALLOW_REGISTER``; if ``True``, any email on any domain can be used to
|
||||
register, if ``False``, only emails which belong to already extant
|
||||
realms can register.
|
||||
realms can register [#]_.
|
||||
|
||||
If ``ALLOW_REGISTER`` is ``False``, MIT users cannot register at all
|
||||
unless they already have an account created via Zephyr mirroring and are
|
||||
activated by us.
|
||||
.. [#]
|
||||
If ``ALLOW_REGISTER`` is ``False``, MIT users cannot register at all
|
||||
unless they already have an account created via Zephyr mirroring and are
|
||||
activated by us.
|
||||
|
||||
Normal user creation
|
||||
~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
#. Visit ``/accounts/home/`` and enter an email address of
|
||||
``<something random>@zulip.com``
|
||||
``<something random>@zulip.com``.
|
||||
#. Check the console where you're running ``run-dev`` for the email, and
|
||||
copy-paste the link, changing the hostname from ``example.com``.
|
||||
#. Fill out the signup form.
|
||||
@@ -272,7 +303,7 @@ You should be sent to the Zulip app after hitting "Register".
|
||||
Realm creation control
|
||||
~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
#. Set ``ALLOW_REGISTER = False``
|
||||
#. Set ``ALLOW_REGISTER = False``.
|
||||
#. Try to sign up with ``alice@example.net``.
|
||||
#. Try to sign up with ``zulip@mit.edu``.
|
||||
|
||||
@@ -281,16 +312,7 @@ You should get an error message for both.
|
||||
MIT user activation
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
#. Set ``ALLOW_REGISTER = False``
|
||||
#. Run ``python manage.py deactivate sipbtest@mit.edu`` (You can use
|
||||
another MIT email address of a Zulip employee here if you'd prefer)
|
||||
#. Run ``python manage.py activate_mit jesstess``. (Replace ``jesstess``
|
||||
with another username as needed)
|
||||
#. Copy over the activation code into your address bar, adjusting the
|
||||
URL as needed.
|
||||
#. Fill out the signup form.
|
||||
|
||||
You should be sent to the Zulip app after hitting "Register".
|
||||
TODO: Do we want to keep this content?
|
||||
|
||||
Mailing list synchronization
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
@@ -304,25 +326,3 @@ configured on your workstation as well as the ``postmonkey`` library.
|
||||
Then, keep ``python manage.py subscribe_new_users`` running while
|
||||
signing up a user and ask somebody to confirm that a user was in fact
|
||||
subscribed on MailChimp. TODO: split tests off into a separate list.
|
||||
|
||||
Test server
|
||||
===========
|
||||
|
||||
There's a test server, test1.zulip.net, for configuration testing only —
|
||||
Apache, Puppet, the ``update-deployment`` script... things you can't
|
||||
test easily on your laptop.
|
||||
|
||||
**test1 uses the production database, as does staging. Do not interact
|
||||
with the production database unless you have a specific reason and
|
||||
people know you are doing it.** In particular, you should **not** run
|
||||
``test-all`` or similar on test1, because this will do things like
|
||||
create/destroy tables which we don't want happening in our production
|
||||
database.
|
||||
|
||||
To deploy on test1, ``cd`` to ``~/zulip-deployments/current`` and run
|
||||
``tools/update-deployment branchname``, where branchname is the branch
|
||||
you want to test.
|
||||
|
||||
You may need to start the server before using it, and you should stop it
|
||||
when you're done. See `Setting up a new
|
||||
server <Setting_up_a_new_server>`__.
|
||||
|
||||
Reference in New Issue
Block a user