paulmataruso/zulip
1
0
Fork 0
mirror of https://github.com/zulip/zulip.git synced 2025-11-06 15:03:34 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
d2e07ea51bb6896ff4ea67152077a5f487ae9ae4
zulip/tools/test-js-with-node
Steve Howell eeee6edf41 pm_list: Simplify redraws for Private Messages. 
We now use vdom-ish techniques to track the
list items for the pm list.  When we go to update
the list, we only re-render nodes whose data
has changed, with two exceptions:

    - Obviously, the first time we do a full render.
    - If the keys for the items have changed (i.e.
      a new node has come in or the order has changed),
      we just re-render the whole list.

If the keys are the same since the last re-render, we
only re-render individual items if their data has
changed.

Most of the new code is in these two modules:

    - pm_list_dom.js
    - vdom.js

We remove all of the code in pm_list.js that is
related to updating DOM with unread counts.

For presence updates, we are now *never*
re-rendering the whole list, since presence
updates only change individual line items and
don't affect the keys.  Instead, we just update
any changed elements in place.

The main thing that makes this all work is the
`update` method in `vdom`, which is totally generic
and essentially does a few simple jobs:

    - detect if keys are different
    - just render the whole ul as needed
    - for items that change, do the appropriate
      jQuery to update the item in place

Note that this code seems to play nice with simplebar.

Also, this code continues to use templates to render
the individual list items.

FWIW this code isn't radically different than list_render,
but it's got some key differences:

    - There are fewer bells and whistles in this code.
      Some of the stuff that list_render does is overkill
      for the PM list.

    - This code detects data changes.

Note that the vdom scheme is agnostic about templates;
it simply requires the child nodes to provide a render
method.  (This is similar to list_render, which is also
technically agnostic about rendering, but which also
does use templates in most cases.)

These fixes are somewhat related to #13605, but we
haven't gotten a solid repro on that issue, and
the scrolling issues there may be orthogonal to the
redraws.  But having fewer moving parts here should
help, and we won't get the rug pulled out from under
us on every presence update.

There are two possible extensions to this that are
somewhat overlapping in nature, but can be done
one a time.

    * We can do a deeper vdom approach here that
      gets us away from templates, and just have
      nodes write to an AST.  I have this on another
      branch, but it might be overkill.

    * We can avoid some redraws by detecting where
      keys are moving up and down.  I'm not completely
      sure we need it for the PM list.

If this gets merged, we may want to try similar
things for the stream list, which also does a fairly
complicated mixture of big-hammer re-renders and
surgical updates-in-place (with custom code).

BTW we have 100% line coverage for vdom.js.
2020-01-30 13:11:32 -08:00

248 lines
8.8 KiB
Python
Executable File
Raw Blame History

#!/usr/bin/env python3
import argparse
import os
import subprocess
import sys
from typing import Dict, Any
TOOLS_DIR = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, os.path.dirname(TOOLS_DIR))
ROOT_DIR = os.path.dirname(TOOLS_DIR)
# check for the venv
from tools.lib import sanity_check
sanity_check.check_venv(__file__)
# Import this after we do the sanity_check so it doesn't crash.
import ujson
INDEX_JS = 'frontend_tests/zjsunit/index.js'
NODE_COVERAGE_PATH = 'var/node-coverage/coverage-final.json'
USAGE = '''
tools/test-js-with-node - to run all tests
tools/test-js-with-node util.js activity.js - to run just a couple tests
tools/test-js-with-node --coverage - to generate coverage report
'''
enforce_fully_covered = {
'static/shared/js/typeahead.js',
'static/shared/js/typing_status.js',
'static/js/activity.js',
'static/js/alert_words.js',
'static/js/alert_words_ui.js',
'static/js/bot_data.js',
'static/js/buddy_data.js',
'static/js/buddy_list.js',
'static/js/channel.js',
'static/js/color_data.js',
'static/js/colorspace.js',
'static/js/common.js',
'static/js/components.js',
'static/js/compose_pm_pill.js',
'static/js/compose_state.js',
'static/js/compose_ui.js',
# Temporarily removed until we finish merging emoji commits.
# 'static/js/composebox_typeahead.js',
'static/js/dict.ts',
'static/js/emoji.js',
'static/js/feature_flags.js',
'static/js/fenced_code.js',
'static/js/fetch_status.js',
'static/js/filter.js',
'static/js/fold_dict.ts',
'static/js/hash_util.js',
'static/js/keydown_util.js',
'static/js/input_pill.js',
'static/js/int_dict.ts',
'static/js/lazy_set.js',
'static/js/list_cursor.js',
'static/js/markdown.js',
'static/js/message_store.js',
'static/js/muting.js',
'static/js/narrow_state.js',
'static/js/people.js',
'static/js/pm_conversations.js',
'static/js/pm_list.js',
'static/js/presence.js',
'static/js/reactions.js',
'static/js/recent_senders.js',
'static/js/rtl.js',
'static/js/schema.js',
'static/js/scroll_util.js',
'static/js/search.js',
'static/js/search_suggestion.js',
'static/js/search_util.js',
'static/js/server_events_dispatch.js',
# Removed because we're migrating code from uncovered other settings pages to here.
# 'static/js/settings_ui.js',
'static/js/settings_muting.js',
'static/js/settings_user_groups.js',
'static/js/stream_data.js',
'static/js/stream_events.js',
'static/js/stream_sort.js',
'static/js/top_left_corner.js',
'static/js/topic_data.js',
'static/js/topic_list_data.js',
'static/js/topic_generator.js',
'static/js/transmit.js',
'static/js/typeahead_helper.js',
'static/js/typing_data.js',
'static/js/unread.js',
'static/js/user_events.js',
'static/js/user_groups.js',
'static/js/user_pill.js',
'static/js/user_search.js',
'static/js/user_status.js',
'static/js/util.js',
'static/js/vdom.js',
'static/js/widgetize.js',
'static/js/search_pill.js',
'static/js/billing/billing.js',
'static/js/billing/upgrade.js',
}
parser = argparse.ArgumentParser(USAGE)
parser.add_argument('--coverage', dest='coverage',
action="store_true",
default=False, help='Get coverage report')
parser.add_argument('--force', dest='force',
action="store_true",
default=False, help='Run tests despite possible problems.')
parser.add_argument('args', nargs=argparse.REMAINDER)
options = parser.parse_args()
individual_files = options.args
from tools.lib.test_script import assert_provisioning_status_ok
assert_provisioning_status_ok(options.force)
def run_tests_via_node_js() -> int:
os.environ['TZ'] = 'UTC'
# Add ".js" to the end of all the file arguments, so index.js
# can actually verify if the file exists or not.
for index, arg in enumerate(options.args):
if not arg.endswith('.js') and not arg.endswith('.ts'):
# If it doesn't end with ".js" or ".ts", assume it is a JS file,
# since most files are currently JS files.
options.args[index] = arg + '.js'
# The index.js test runner is the real "driver" here, and we launch
# with either nyc or node, depending on whether we want coverage
# reports. Running under nyc is slower and creates funny
# tracebacks, so you generally want to get coverage reports only
# after making sure tests will pass.
node_tests_cmd = ['node', '--stack-trace-limit=100', INDEX_JS]
node_tests_cmd += individual_files
if options.coverage:
coverage_dir = os.path.join(ROOT_DIR, 'var/node-coverage')
coverage_lcov_file = os.path.join(coverage_dir, 'lcov.info')
nyc = os.path.join(ROOT_DIR, 'node_modules/.bin/nyc')
command = [nyc, '--extension', '.ts']
command += ['--report-dir', coverage_dir]
command += ['--temp-directory', coverage_dir, '-r=text-summary']
command += node_tests_cmd
command += ['&&', 'nyc', 'report', '-r=lcov', '-r=json']
command += ['>', coverage_lcov_file]
else:
# Normal testing, no coverage analysis.
# Run the index.js test runner, which runs all the other tests.
command = node_tests_cmd
print('Starting node tests...')
# If we got this far, we can run the tests!
try:
ret = subprocess.check_call(command)
except OSError:
print('Bad command: %s' % (command,))
raise
except subprocess.CalledProcessError:
print('\n** Tests failed, PLEASE FIX! **\n')
sys.exit(1)
return ret
def check_line_coverage(fn, line_coverage, line_mapping, log=True):
# type: (str, Dict[Any, Any], Dict[Any, Any], bool) -> bool
missing_lines = []
for line in line_coverage:
if line_coverage[line] == 0:
actual_line = line_mapping[line]
missing_lines.append(str(actual_line["start"]["line"]))
if missing_lines:
if log:
print("ERROR: %s no longer has complete node test coverage" % (fn,))
print(" Lines missing coverage: %s" % (", ".join(sorted(missing_lines, key=int)),))
print()
return False
return True
def read_coverage() -> Any:
coverage_json = None
try:
with open(NODE_COVERAGE_PATH, 'r') as f:
coverage_json = ujson.load(f)
except IOError:
print(NODE_COVERAGE_PATH + " doesn't exist. Cannot enforce fully covered files.")
raise
return coverage_json
def enforce_proper_coverage(coverage_json: Any) -> bool:
coverage_lost = False
for relative_path in enforce_fully_covered:
path = ROOT_DIR + "/" + relative_path
if not (path in coverage_json):
print("ERROR: %s has no node test coverage" % (relative_path,))
continue
line_coverage = coverage_json[path]['s']
line_mapping = coverage_json[path]['statementMap']
if not check_line_coverage(relative_path, line_coverage, line_mapping):
coverage_lost = True
if coverage_lost:
print()
print("It looks like your changes lost 100% test coverage in one or more files.")
print("Usually, the right fix for this is to add some tests.")
print("But also check out the include/exclude lists in `tools/test-js-with-node`.")
print("To run this check locally, use `test-js-with-node --coverage`.")
coverage_not_enforced = False
for path in coverage_json:
if '/static/js/' in path or '/static/shared/js' in path:
relative_path = os.path.relpath(path, ROOT_DIR)
line_coverage = coverage_json[path]['s']
line_mapping = coverage_json[path]['statementMap']
if check_line_coverage(relative_path, line_coverage, line_mapping, log=False) \
and not (relative_path in enforce_fully_covered):
coverage_not_enforced = True
print("ERROR: %s has complete node test coverage and is not enforced." % (relative_path,))
if coverage_not_enforced:
print()
print("There are one or more fully covered files that are not enforced.")
print("Add the file(s) to enforce_fully_covered in `tools/test-js-with-node`.")
problems_encountered = (coverage_lost or coverage_not_enforced)
return problems_encountered
ret = run_tests_via_node_js()
if options.coverage and ret == 0:
if not individual_files:
coverage_json = read_coverage()
problems_encountered = enforce_proper_coverage(coverage_json)
if problems_encountered:
ret = 1
print()
if ret == 0:
if options.coverage:
print("View coverage reports at http://127.0.0.1:9991/node-coverage/index.html")
print("Test(s) passed. SUCCESS!")
else:
print("FAIL - Test(s) failed")
sys.exit(ret)
Reference in New Issue View Git Blame Copy Permalink