Like the recent change blocking JSON endpoints for deactivated users
and users in deactivated realms, this change is a hardening
improvement. Those users should be unable to get an active session
anyway, but if somehow one is leaked, this means they won't be able to
access any user data.
Previously, api_fetch_api_key would not give clear error messages if
password auth was disabled or the user's realm had been deactivated;
additionally, the account disabled error stopped triggering when we
moved the active account check into the auth decorators.
While in theory users should be unable to get a valid session in order
to access these endpoints in the first place, this provides an extra
layer of hardering to prevent a deactivated user with a session from
accessing data via the old-style JSON API.
The security model for deactivated users (and users in deactivated
realms) being unable to access the service is intended to work via two
mechanisms:
* All active user sessions are deleted, and all login code paths
(where a user could get a new session) check whether the user (or
realm) is inactive before authorizing the request, preventing the
user from accessing the website and AJAX endpoints.
* All API code paths (which don't require a session) check whether the
user (and realm) are active.
However, this security model was not implemented correctly. In
particular, the check for whether a user has an active account in the
login process was done inside the login form's validators, which meant
that authentication mechanisms that did not use the login form
(e.g. Google and REMOTE_USER auth) could succeed in granting a session
even with an inactive account. The Zulip homepage would still fail to
load because the code for / includes an API call to Tornado authorized
by the user's token that would fail, but this mechanism could allow an
inactive user to access realm data or users to access data in a
deactivated realm.
This fixes the issue by adding explicit checks for inactive users and
inactive realms in all authentication backends (even those that were
already protected by the login form validator).
Mirror dummy users are already inactive, so we can remove the explicit
code around mirror dummy users.
The following commits add a complete set of tests for Zulip's inactive
user and realm security model.
In a deactivated realm, webhooks would still successfully send
messages, since there was no check for whether the realm was active in
api_key_only_webhook_view.
Long ago, there was work on an experimental integration model where
every user in a realm would have administrative control over all bots,
with the goal of simplifying the process of setting up communally
administered bots for smaller teams. While that new model was never
fully implemented (and thus never setup as an option), an error in
that original implementation meant that the data on all bots in a
realm, including their API keys, was sent to the browsers of users via
the `realm_bots` variable in `page_params`. The data wasn't displayed
in the UI for non-admin users, but was available via e.g. the
javascript console.
This commit updates this behavior to only send sensitive bot data like
API keys to the owner of the bot (and realm admins).
We may in the future implement a model simplifying communally
administered integrations, but if we do that, those bots should be
limited in their capabilities (e.g. only able to send webhook
messages).
This bug has been present since Zulip was released as open source.
The old code for this lookup was unnecessarily complicated because we
were working around Guardian, where the `is_realm_admin` check was
extremely expensive.
Previously we relied on having two matching list of fields for the
get_active_user_dicts_in_realm, one in the actual code and the other
in the caching system. By unifying these lists to have a single
source, we eliminate a class of caching bugs we might otherwise
regularly introduce.
Camo is a caching image proxy, used in Zulip to avoid mixed-content
warnings by proxying HTTP image content over HTTPS. We've been using
it in zulip.com production for years; this change makes it available
in standalone Zulip deployments.
This should save several minutes off the Travis CI `production`
suite's runtime, since previously we were doing the full apt upgrade
process twice, resulting in things like multiple expensive rebuilds of
the initramfs.
Add 'six' to setup-py3k, because it is being used in tools/lister.py.
Add 'typing' to setup-py3k, so that tools/lister.py can be type
annotated in the future.
Since we don't have a stable way to get the Dropbox preview failure
image (and it was sorta a weird setup anyway), it seems best to just
remove the condition.
Previously, the user list would remain filtered after a user hit enter
to start composing a message to a user, leaving them in a state with a
partial user list.
Fixes#360.
Previously we needed to use a specified password when activating a
formerly mirror dummy user, in order for that user to be able to
(re)set their password and login. Now that we have our own password
reset form, this is no longer required.
Previously, if a user had only authenticated via Google auth, they
would be unable to reset their password in order to set one (which is
needed to setup the mobile apps, for example).
Now tools/check-py3 will by default run all fixers together. This is
quicker but doesn't indicate which fixers caused the failure. The
newly added option --find-fixers falls back to the old way of checking
each fixer separately if the quick check fails.
Fixes#710.
Apparently LXC 2 removed support for the `-B best` option in
lxc-create, and Vagrant hasn't been updated appropriately yet, so we
need to add a workaround to explicitly specify a backing store.
Fixes#718.
This manifested as errors of the form:
"""
There was an error executing ["sudo",
"/usr/local/bin/vagrant-lxc-wrapper", "lxc-create", "-B", "best",
"--template",
"/home/tabbott/.vagrant.d/gems/gems/vagrant-lxc-1.2.1/scripts/lxc-template",
"--name", "zulip_default_1461801696512_85064", "--", "--tarball",
"/home/tabbott/.vagrant.d/boxes/fgrehm-VAGRANTSLASH-trusty64-lxc/1.2.0/lxc/rootfs.tar.gz",
"--config",
"/home/tabbott/.vagrant.d/boxes/fgrehm-VAGRANTSLASH-trusty64-lxc/1.2.0/lxc/lxc-config"]
"""
Several recently merged webhooks were incorrectly not checking that
the actual webhook result didn't return an error. While they would
usually still fail in most cases when checking whether the message
came back correctly, this hid the root cause errors and thus made it
much harder to debug.
We were incorrectly applying the rate limiting rules to webhooks even
if rate limiting was disabled (as in the test suite), causing test
failures when the total number of webhook tests in Zulip got too high.
In theory, tools like populate_db should probably be in zerver, not
zilencer, but until we migrate them out, we need to include these in
EXTRA_INSTALLED_APPS in development.
The previous separated-out configuration wasn't helping us, and this
makes it easier to make the extra installed applications pluggable in
the following commits.
This will merge conflict with every new integraiton in flight, which
is unfortunate, but will make there be fewer merge conflicts as people
add new webhooks in the future (currently, every pair of new
integrations conflict because folks are adding them all at the end,
whereas after this change, there will only be merge conflicts when
adding two integrations near each other alphabetically).
This integration relies on the Teamcity "tcWebHooks" plugin which is
available at
https://netwolfuk.wordpress.com/category/teamcity/tcplugins/tcwebhooks/
It posts build fail and success notifications to a stream specified in
the webhook URL.
It uses the name of the build configuration as the topic.
For personal builds, it tries to map the Teamcity username to a Zulip
username, and sends a private message to that person.
As documented in https://github.com/zulip/zulip/issues/441, Guardian
has quite poor performance, and in fact almost 50% of the time spent
running the Zulip backend test suite on my laptop was inside Guardian.
As part of this migration, we also clean up the old API_SUPER_USERS
variable used to mark EMAIL_GATEWAY_BOT as an API super user; now that
permission is managed entirely via the database.
When rebasing past this commit, developers will need to do a
`manage.py migrate` in order to apply the migration changes before the
server will run again.
We can't yet remove Guardian from INSTALLED_APPS, requirements.txt,
etc. in this release, because otherwise the reverse migration won't
work.
Fixes#441.
When uploaded avatar image is not a valid image file, PIL raises
IOError. Catch the IOError raised by PIL and raise JsonableError.
This will return a response with status code 400.
S3Test is now only the S3-specific test (which isn't even run), so we
can now invest in making FileUploadTest have good coverage of the
(local) file upload code paths.
The main function of prompting inside `manage.py migrate` is to ask
the user if they want to delete stale content-types, which is
unimportant and likely scary, so we disable doing so.
Previously, the UserProfile objects were created in the order
generated by a Set, which meant tests would randomly start failing if
the code that runs before this part of populate_db changed (and thus
caused the Set object used to pass users into bulk_create_users to
have a different order when enumerated).
This fixes the issue in two ways -- one by sorting the users inside
bulk_create_users, and second by attaching subscriptions to users
based on a deterministic ordering.
The restarted Tornado processes seemed to escape the process group and
thus continue running after run-dev.py finished.
While we're at it, we don't need to dump/reload event queues in the
test suite either.
Move recenter_pointer_on_display, suppress_scroll_pointer_update,
fast_forward_pointer, furthest_read, and server_furthest_read to
a new pointer module in pointer.js.
The `with sh.sudo` pattern that we were using in python-sh was
deprecated, and emperically hangs on Ubuntu xenial. Since in general
the use of python-sh/python-pbs caused trouble (requiring extra
dependencies, confusing syntax), this just removes it.
We replace it with a new zulip_tools.py library function that echoes
the command line and streams the output.
We do the same to install-phantomjs so we can remove that dependency.
Previously, the Zulip subscriptions page's error bar would always be
at the very top of the scrollable view, and thus would likely be out
of view when an error happened. This fixes it by having the error bar
always placed below the search box (and thus visible regardless of
where in the scrollable streams view we are).
Fixes: #515.
[commit message and comments expanded by tabbott]
This reverts commit d936bf61f9.
We no longer need this since we've migrated to specifying the
dependencies in the typing module that we're actually using.
This fixes an issue where this worker wasn't even being installed
properly in a way that sets us up for doing further reorganization of
the Zulip Nagios plugins.
tools/travis/py3k used to always exit with exit code 0.
It should exit with 1 when fixers detect a compatibility issue.
py3k used [ -z "$failed" ] to check if there was a failure.
This is wrong, since if no failure has occured, failed=0,
and -z checks if a string is of zero length. This commit also
fixes this bug.
In py3k, "git reset --hard" was called only if
libmodernize.fixes.fix_dict_six changed files and some of those
changes are not considered false positives by py3k.
But if all of those changes are not considered false positives
by py3k, then "git reset --hard" is not called and the repository
is no longer clean.
This commit fixes this bug.
tools/travis/py3k used to only check files whose names ended with .py.
Now it also checks python scripts which don't have an extension.
It uses tools/lister.py to get a list of all python files.
The previous version of sanitize_name dropped all unicode characters
and mangled filenames with multiple `.`s in the extension, leading to
confusing URLs for files uploaded to Zulip.
Fixes#321.
[tweaked significantly by tabbott]
This has the side effect of making lint-all check all shell scripts,
not just those under scripts/, tools/, and bin/.
[commit message expanded by tabbott]
Make module tools/lister.py which lists all files in a directory
tracked by git. This is done because lister.py will be used by other
scripts in the future which have to introspect files in the repository,
like linters, static code checkers, etc.
It's always been the case that in production, Tornado dumps all the
event queues when shut down so that they can be reloaded by the
replacement Tornado process. This never worked in development because
the codepath for auto-reload didn't go through either a signal or
sys.exit (it re-execs the process instead).
This meant that we didn't have a mechanism for testing the event queue
dump/load functionality in the development environment. We fix this
by adding such dumping/loading. However, this breaks the automatic
reloading of open browser windows on a server restart, so we add that
back in by adjusting the special `restart` events to pass a special
`immediate` flag when used in development.
This also has the benefit of removing the "Bad event queue" errors one
would get on every file save induced restart on the Python console.
Apparently it isn't always the case that removal of jquery and the DOM
prevents cleanup_event_queue from being called via the postunload
hook, so add a check to avoid it being double-called.
Previously, the browser might restart a get_events operation even
while it was in the middle of executing a `DELETE /events` query to
cause its event queue to be de-allocated. This was a rare race
condition when we weren't notifying clients when event queues were
de-allocated, but this will become a common case in the next commit.
This is a no-op right now, but we'll want the new structure for the
next commit, and splitting this out makes it a lot easier to read what
is actually changed in the next commit.
This should fix a problem we've been having with errors downloading
the PhantomJS packages from their original hosting service.
Eventually we should move it to an S3 bucket.
cd2348e9ae broke installing Zulip in
production since it didn't correctly update the puppet configuration
to call the process_queue script using the new argument format.
This commit isn't ideal in that I'd prefer to not require updating
puppet in sync with the actual running code, but we don't have a great
mechanism for doing that.
Fixes#586.
Previously, we needed to update the installation instructions with the
current version of Zulip in production every time we did a release,
which was kinda a pain (and hadn't happened since 1.3.6).
Fixes#576.
[commit message details expanded by tabbott]
The original logic for incremental presence list updating from
668d0d9dfa incorrectly attempted to
insert the user 1 spot later than its proper index in the listing.
Now that we're doing presence updates in a performant fashion, we
don't need to throttle processing these events, and in fact the
throttling of these events created a correctness problem, since we're
now doing incremental updates rather than just rerendering everything
after each event.
The code in 668d0d9dfa for removing an
existing user from the user list to update the status didn't correctly
quote the email address of the user in its jquery selector.
While we already don't link to /terms anywhere on the site, they can still be
accessed if you navigate to /terms directly. Now, those routes will only be
exported on the Zulip.com service.
We should ideally provide a mechanism for deployments to specify their own
terms without modifying source code; in the interim, sites that have already
customised the provided Zulip.com terms can simply carry a patch reverting this
commit.
This change drops the memory used for Python processes run by Zulip in
development from about 1GB to 300MB on my laptop.
On the front of safety, http://pika.readthedocs.org/en/latest/faq.html
explains "Pika does not have any notion of threading in the code. If
you want to use Pika with threading, make sure you have a Pika
connection per thread, created in that thread. It is not safe to share
one Pika connection across threads.". Since this code only connects
to rabbitmq inside the individual threads, I believe this should be
safe.
Progress towards #32.
First user-fasing problem is that when user click to "Collapse" button
of message from narrowed list, buttons "Uncollapse" and "[More...]" does
not work. Second, is that when user collapse/uncollapse some message
from narrowed list, the collapsing/uncollapsing of the same message in
home list does not work in appropriate way.
In "popovers.js" there is the function that is called on click to the
buttons "Collapse" or "Un-collapse". It should show and hide body of a
message. If a message list is narrowed, it should show/hide message in
home list too. So, the first problem is that "toggle_row()" in this
function call methods "collapse(row)" or "uncollapse(row)" from
"condense.js" twice (for row and home_row) using condition
"if (message.collapsed)". When it happen the first time, the variable
"message.collapsed" is changed. That is why next call of "toggle_row()"
work incorrectly.
The second problem is that the function in "condense.js" that is
called on click to the button "[More...]" contains no code for
collapsing/uncollapsing message from home list. It just calls
"collapse(row)" or "uncollapse(row)" for row from narrowed list.
Now, functions "collapse(row)" and "uncollapse(row)" get row from
current list and change both messages (from current list and home
list). On-click functions call them just once for making all of needed
message changes. So, when user collapse or uncollapse message from
home or narrowed list it works correctly.
Fixes: #516
Apparently, our event queue garbage collection logic never actually
disconnected any existing handler objects.
We fix this by disconnecting the handler inside cleanup(), adding a
special check to avoid creating a pointless timeout object.
The new Tornado handler tracking logic properly handled requests that
threw an exception or followed the RespondAsynchronously code path,
but did not properly de-allocated the handler in the syncronous case.
An easy reproducer for this is to load a new Zulip browser window;
that will leak 2 handler objects for the 2 synchronous requests made
from Django to Tornado as part of initial state fetching.
This line appears to have been lost in rebasing from the original
implementation of 1396eb7022faec4c2d91553800a35781a96dd5bd; so the
previous fix actually only addressed the issue in a rare exception
case.
The recent Tornado memory leak fix
(1396eb7022) didn't use the correct
variable name for the current handler ID, causing this cleanup code to
fail in the event that a view raised an exception.
Replaced calls to ifilterfalse by list comprehensions because
ifilterfalse is not part of python 3. Also changed some lists to sets
for faster lookup.
Refer to #256.
This automatically loads settings, zerver.models.* and
zerver.lib.actions.* when you start `manage.py shell`, which should
save a bit of time basically every time someone uses it.
Fixes#275.
Previously, even though the Zulip digest emails were documented in the
settings, the cron job to run the script that actually sends the daily
digest emails wasn't included in the non-zulip.com part of the Zulip
production distribution. The overall consequence is that digest
emails didn't work for non-zulip.com users. This fixes that issue by
moving that cron job into the zulip manifests.
[commit message details expanded by tabbott]
In 2ea0daab19, handlers were moved to
being tracked via the handlers_by_id dict, but nothing cleared this
dict, resulting in every handler object being leaked. Since a Tornado
process uses a different handler object for every request, this
resulted in a significant memory leak. We fix this by clearing the
handlers_by_id dict in the two code paths that would result in a
Tornado handler being de-allocated: the exception codepath and the
handler disconnect codepath.
Fixes#463.
Saving the organization settings form in the administration did not
work due to a trivial form name mismatch caused by following
revisions: 472898c and 58aba59.
The only places we use the architecture were for finding the
tsearch_extras and phantomjs binaries; Luke Faraone kindly uploaded
both 32-bit and 64-bit binaries for tsearch_extras 0.1.3, so with a
bit of refactoring, we can now support 32-bit.
Fixes#505.
Add call to tools/generate-fixtures in tools/test-backend before
starting the tests. Previously, test-backend could fail if called
after tools/test-js-with-casper had failed.
Fixes#501.
Previously, we used shell quoting that would result in the shell variable not
being substituted. Instead, we use `"`s that will allow for variable
substitution.
We also explicitly include `ca-certificates`, as it is needed for the install
to complete. Usually this is brought in as a `Recommends` of `wget`, but some
systems may not automatically include such dependencies.
Fixes#470.
Whenever a user became active, this triggers an immediate presence
update event (to show that user as active). The implementation for
that event (running on the browsers of all other users in the realm)
would fully rerender the presence list, which can be an expensive
operation in a large realm, just to update the status for that one
user. This fixes that case to just remove the user from the list and
then re-insert it at the appropriate index.
[Commit message expanded with more details by Tim Abbott]
Add a function email_allowed_for_realm that checks whether a user with
given email is allowed to join a given realm (either because the email
has the right domain, or because the realm is open), and use it
whenever deciding whether to allow adding a user to a realm.
This commit is not intended to change any behavior, except in one case
where the Zulip realm's domain was not being converted to lowercase.
manage_args is set to a list of arguments a few lines later in the
function, making this initialization as the empty string useless and
confusing.
Discovered using mypy.
While I believe this actually produced correct output since users are
always subscribed to streams within their realm, this code was
definitely wrong.
Discovered using the mypy type-checking tool.
It's not clear whether this will end up being net negative in value in
the long term since it's kinda hard to understand the output, but in
the short term it should prevent regressions.
If the content wasn't rendered, both rendered_content and
rendered_content_version would be None. In addition to being
confusing, in Python 3, `None < 2` is an error and this code breaks.
At present, we only do a few simple checks on the client type inside
the event system, and this saves database/memcached queries.
Note that this preserves the structure of the marshalled name in
to_dict/from_dict as client_type to avoid an unnecessary migration.
Previously, client descriptors were referenced directly from the
handler object. Once we split the Tornado process into separate queue
and connection servers, these will no longer be in the same process,
so we need to reference them by ID instead.
This commit is somewhat ugly, but its purpose is to be early
preparation for splitting Tornado into a queue server and a frontend
server, and this code belongs, by and large, in the queue server
component.
89a2765553 didn't include the database
migration corresponding to the change, which means it didn't take full
effect when it was merged.
I noticed this because `manage.py makemigrations` would generate these
migrations; that suggests a good idea for a test to add.
This solves the problem reported in #331 with needing to specify
--provider=lxc to use the LXC provider in an Ubuntu Linux environment;
additionally, it adds the LXC option needed to run LXC on Ubuntu
15.10, but not on 14.04 where that option is unavailable and would
totally break LXC.
It's possible we should just eliminate this mechanism, but this fixes
a proximal problem where the multi-line get_subscribers endpoint
description was being handled wrong.
Previously these were hardcoded in zproject/settings.py to be accessed
on localhost.
[Modified by Tim Abbott to adjust comments and fix configure-rabbitmq]
Travis CI's model of installing every version of postgres on the test
VM and then shutting all the versions other than the one requested
down seems to not work very well with doing apt upgrades. It seems
the best way to resolve this is to just uninstall the versions we
don't need.
A common issue when doing a Zulip upgrade is trying to pass
upgrade-zulip a tarball path under /root, which doesn't work because
the Zulip user doesn't have permission to read the tarball. We
could fix this by just unpacking the tarballs as root, but it seemed
like a nicer approach would be to archive the release tarballs
somewhere readable by the Zulip user (/home/zulip/archives) and unpack
them from there.
Fixes#208.
The point of the lock is to prevent two deployments happening at the
same time and racing with each other, not to prevent doing any future
deployments after an error happens (which is what the current
implementation does in practice).
Addresses part of #208.
This link was broken when we hardened the access model for user file
uploads to not work cross-realm. The right solution is just to
include the image in the codebase so it's guaranteed to exist.
Fixes#205.
Previously:
* It wouldn't raise an exception if the stream didn't exist
* It didn't correctly handle being passed a stream name
that differed in case from the stream name in the database.
The previous implementation didn't work because HomepageForm rejected
the email as not having a domain. Additionally, the logic in
accounts_register didn't work with Google auth because that code path
doesn't pass through accounts_home. Since whether there's a unique
open realm for the server is effectively a configuration property, we
can fix the bug and make the logic clearer by moving it into the
"figure out the user's realm" function.
The browser registers for events via loading the home view, not this
interface, and this functionality is available via the API-format
register route anyway.
This removes from our cache a moderate amount of totally useless alert
word data corresponding to users who don't have any alert words.
Thanks to @dbiollo for the suggestion!
Just doing the database query is more readable, and has about the same
performance as before in the case where active user dicts for the
realm are in cache (and is substantially better in the rare case that
this isn't in the cache).
Thanks to @dbiollo for the perf investigation and suggestion!
This makes it possible to use DevAuthBackend when doing
performance/scalability testing on Zulip with many thousands of users.
It's unlikely that anyone testing this backend will find it valuable
to have more than 100 login buttons on the same page, and if they do,
they can always just change this limit.
Thanks to @dbiollo for the suggestion!
This fixes a performance issue looking up UserProfile objects for
realms with a large number of users in the case that a UserProfile
object is not in the cache.
Thanks to @dbiollo for the suggestion!
We ran into a bug with the Travis CI infrastructure where it postgres
9.1 is installed on the system, and so when we'd do an apt upgrade
with a new version of 9.1, the 9.1 daemon would end up getting started
and conflict with the 9.3 daemon we were trying to run.
860cf68716 introduced calls to
notifications.redraw_title() on narrow activation. This introduced a
bug when the Zulip desktop app reloads while narrowed --
new_message_count would still be set to undefined when
narrow.activate() is called as the page (re)loads, and thus we'd call
window.bridge.updateCount(undefined), resulting in a traceback.
We fix this by just initializing it to 0, rather than using the old
default value of undefined.
This allows full-screen mode when launching from a saved app link
(mobile browser -> save link to home screen). This works on Android,
too, despite the "apple-" prefix.
Django's `manage.py runserver` prints a relatively low-information log
line for every request of the form:
[14/Dec/2015 00:43:06]"GET /static/js/message_list.js HTTP/1.0" 200 21969
This is pretty spammy, especially given that we already have our own
middleware printing a more detailed version of the same log lines:
2015-12-14 00:43:06,935 INFO 127.0.0.1 GET 200 0ms /static/js/message_list.js (unauth via ?)
Since runserver doesn't have support controlling whether these log
lines are printed, we wrap it with a small bit of code that silences
the log lines for 200/304 requests (aka the uninteresting ones).
It's needed for the tornado server. Otherwise, you get errors like
2015-12-20 09:33:55,124 ERROR Internal Server Error: /api/v1/events
Traceback (most recent call last):
File "/home/zulip/deployments/2015-12-20-13-44-47/zerver/management/commands/runtornado.py", line 209, in get_response
response = middleware_method(request)
File "/usr/lib/python2.7/dist-packages/django/middleware/common.py", line 62, in process_request
host = request.get_host()
File "/usr/lib/python2.7/dist-packages/django/http/request.py", line 101, in get_host
raise DisallowedHost(msg)
DisallowedHost: Invalid HTTP_HOST header: 'localhost:9993'. You may need to add u'localhost' to ALLOWED_HOSTS.
Like the Stream Subject lists, Private messages are now shown
when the user clicks on the "Private message" link. User can drill in
to get more than 5 conversations. Selecting PMs from the user or group
PM lists on the right sidebar also opens the list & highlights the
selected conversation.
[Edited by tabbott@mit.edu to fix some small bugs.]
Several of these rules only apply to one of Python and Javascript, and
this simplifies the logic and should make our linter code more readable.
In the process, we add support for per-rule/file pair exclusions to
handle the tab exception for codehilite.py.
By default we are placed inside a virtualenv by the .bash_profile using
/usr/bin/python forces the provisioning script to run outside of this
virtualenv.
These routes previously didn't follow our standard convention of
sending arguments in JSON format, and so broke when we started
checking the argument format in
123d51e3aa.
Fixes#333.
Apparently, previously nginx was only compressing text/html content.
This should result in a substantial savings in network traffic -- some
quick testing I did found it cut the total data transferred for
loading a logged-in zulip.com instance from 3MB to 1.2MB.
`tools/test-js-with-node cover` needs istanbul to be installed in
order to work; we might as well install it by default rather than
having it be an extra step users need to deal with.
Of course, since this is only needed in the development environment,
this could suggest we want to fork/conditionalize package.json, but I
think for now it's reasonable to just install everything we use
somewhere -- the npm list is still pretty short and we have that issue
anyway with webpack-dev-server.
If running on Django 1.8, running these plugins would die with the below. A fix
for this is to run `django.setup()` before interacting with Django.
Refs:
https://docs.djangoproject.com/en/1.8/ref/applications/#troubleshooting
```
Traceback (most recent call last):
File "/usr/lib/nagios/plugins/check_send_receive_time", line 103, in <module>
sender = get_user_profile_by_email(settings.NAGIOS_SEND_BOT)
File "/home/zulip/deployments/current/zerver/lib/cache.py", line 113, in func_with_caching
val = func(*args, **kwargs)
File "/home/zulip/deployments/current/zerver/models.py", line 1073, in get_user_profile_by_email
return UserProfile.objects.select_related().get(email__iexact=email.strip())
File "/usr/lib/python2.7/dist-packages/django/db/models/query.py", line 328, in get
num = len(clone)
File "/usr/lib/python2.7/dist-packages/django/db/models/query.py", line 144, in __len__
self._fetch_all()
File "/usr/lib/python2.7/dist-packages/django/db/models/query.py", line 977, in _fetch_all
self._result_cache = list(self.iterator())
File "/usr/lib/python2.7/dist-packages/django/db/models/query.py", line 238, in iterator
results = compiler.execute_sql()
File "/usr/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 829, in execute_sql
sql, params = self.as_sql()
File "/usr/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 378, in as_sql
extra_select, order_by, group_by = self.pre_sql_setup()
File "/usr/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 48, in pre_sql_setup
self.setup_query()
File "/usr/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 39, in setup_query
self.select, self.klass_info, self.annotation_col_map = self.get_select()
File "/usr/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 206, in get_select
related_klass_infos = self.get_related_selections(select)
File "/usr/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 700, in get_related_selections
[f.name], opts, root_alias)
File "/usr/lib/python2.7/dist-packages/django/db/models/sql/query.py", line 1471, in setup_joins
names, opts, allow_many, fail_on_missing=True)
File "/usr/lib/python2.7/dist-packages/django/db/models/sql/query.py", line 1372, in names_to_path
if field.is_relation and not field.related_model:
File "/usr/lib/python2.7/dist-packages/django/utils/functional.py", line 60, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "/usr/lib/python2.7/dist-packages/django/db/models/fields/related.py", line 110, in related_model
apps.check_models_ready()
File "/usr/lib/python2.7/dist-packages/django/apps/registry.py", line 131, in check_models_ready
raise AppRegistryNotReady("Models aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Models aren't loaded yet.
```
Primarily this makes sure all the log files we generate are ignored.
A good follow-up project to this would be to move all the log files to
a fixed directory so that they're not creating a mess in the main
filespace.
The #! line processing interpreted the argument to pass to `env` as
"python2.7 -u", which obviously isn't a real program.
We fix this by setting the PYTHONUNBUFFERED environment variable
inside the program, which has the same effect.
Thanks to Dan Fedele for the bug report and suggested solution!
* Reorganize to cover how to use the VM regardless of install process
used.
* Document exactly what you need to do in order to see your changes.
* Remove the now-inaccurate documentation about flaky casper tests.
* Point to the testing documentation.
This test caught a few bugs where refactoring had made management
commands fail (and would have caught a few more recent ones).
Ideally we'd replace this with a more advanced test that actually
tests that the management command do something useful, but it's a
start.
notify_new_user was recently moved to zerver.lib.actions from
zerver.views and this wasn't properly updated. This would give an
error when doing a `manage.py create_user` from the command line.
The SSO build of the desktop app is intended only for those users who
who have settings.SSO_ONLY set, i.e. the only way to login is via the
site's SSO REMOTE_USER authentication. We were incorrectly linking to
it on all production installations :(.
The node packages 'jQuery' and 'jquery' are different--'jQuery' is the
legacy support package that is needed for Zulip so the require statements
in the tests were updated.
Travis uses node 4.0 by default and we are using 0.10, so the command to
install the correct version had to be added to the .travis.yml file.
Some dependencies aren't configured to find the node binary correctly on
Debian (since it is called nodejs instead of node). The node-legacy package
fixes this.
The previous code was using the same codepath as for real users, which
was unfortunate in two ways:
* It hit the wrong endpoint on the server and thus failed
* It popped up the "remove a user prompt" which described a bunch of
things not relevant to bots.
Because the `owner` field had the class email, we were sending the
concatination of the user and owner email addresses as the email
address in the reactivate requests.
Fixes#243.
This tests whether a new patch introduces any regressions related to
any of the Python 3 compatibility fixers we've run in the past, so
that we can make continuous forward progress on our path towards
Python 3 compatibility.
This produces error output that looks like this:
"""
Testing for additions of Python 2 patterns we've removed as part of moving towards Python 3 compatibility.
Running Python 3 compatibility test lib2to3.fixes.fix_apply
Running Python 3 compatibility test lib2to3.fixes.fix_except
diff --git a/zerver/views/__init__.py b/zerver/views/__init__.py
index b5c0102..2defd46 100644
--- a/zerver/views/__init__.py
+++ b/zerver/views/__init__.py
@@ -296,7 +296,7 @@ def accounts_register(request):
do_activate_user(user_profile)
do_change_password(user_profile, password)
do_change_full_name(user_profile, full_name)
- except UserProfile.DoesNotExist, e:
+ except UserProfile.DoesNotExist as e:
user_profile = do_create_user(email, password, realm, full_name, short_name,
prereg_user=prereg_user,
newsletter_data={"IP": request.META['REMOTE_ADDR']})
Python 3 compatibility error(s) detected! See diff above for what you need to change.
"""
In order to enable internationalization support in Zulip, and to use
Django internationalization tools, all strings in Zulip frontend needs
to be marked for translation.
Running check-templates test fails when there are 'blocktrans' tags in
django templates. The fix is to add 'blocktrans' to
is_django_block_tag function in check-templates.
With this change, we are now testing the production static asset
pipeline and installation process in a new testing job (and also run
the frontend/backend tests separately).
This means that changes that break the Zulip static asset pipeline or
production installation process are more likely to fail tests. The
testing is imperfect in that it does not have proper isolation -- we
build a complete Zulip development environment and then install a
Zulip production environment on top of it, so e.g. any apt
dependencies installed for Zulip development will still be available
for the Zulip production environment. But, it's better than nothing!
A good v2 of this would be to have the production setup process just
install the minimum stuff needed to run `build-release-tarball` and
then uninstall it / clean it up so that we can do a more clear
production installation, but that's more work.
This fixes an annoying issue where one tries to rebuild the database,
and it fails due to there being existing connections.
The one thing that is potentially scary about this implementation is
that it means it's now a lot easier to accidentally drop your
production database by running the wrong script; might be worth adding
a "--force" flag controlling this behavior or something.
Thanks to Nemanja Stanarevic and Neeraj Wahi for prototypes of this
implementation! They did most of the work and testing for this.
The --insecure option ("insecure=true" in .zuliprc) disables
verification entirely, similar to other tools like curl.
The --cert_bundle ("cert_bundle=<file>" in .zuliprc) allows
a file to be specified containing the CA certificates to verify
against.
When using self-signed certificates, the server's public key
can be used as the only cerificate in the file.
This change incidentally fixes an issue where the "site" parameter
in .zuliprc was ignored when specifying --user and --api-key on
the command line.
Fixes: #104
Prior to adding reply-to-missed-message-email functionality, adding
automated tests for simpler case - incoming stream messages. Added
to new file test_email_mirror.py.
Also removed the "if not body" code from process_stream_message that
will never run because of an upstream ZulipEmailForwardError exception.
Previously these users' names were being set to 1-element lists
containing the name, not the names themselves. This bug caused
existing users to have their people module state (e.g. @-mentions,
etc.) to break whenever a new user joined.
Fixes#222.
The tests to recognize a misbehaving/unpredictable worker task use
the words 'flaky' and 'freaking out' in personifying the system
behavior. This terminology isn't inclusive of people with
mental health issues or mood disorders, so this change updates
the wording to have less personification and more objective system
description. (http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1925070/)
New users will no longer get desktop and audible notifications for all streams
by default.
This also updates the `day1` follow-up email to let users know they can
customize how and when Zulip notifies them of new messages.
Lastly, this adds a `changelog.md` file, following the conventions from
keepachangelog.com, to track changes for new releases.
This fixes some issues that we've had where commands will fail is
confusing ways after the database is rebuilt because data from before
the database was dropped is still in the memcached cache.
get_realm is better in two key ways:
* It uses memcached to fetch the data from the cache and thus is faster.
* It does a case-insensitive query and thus is more safe.
This addresses a few issues:
* The LDAP authentication integration now creates an account a new
Zulip account if the user authenticated correctly but didn't have a
Zulip account.
* The previous code didn't correctly disable the LDAP group
permissions functionality. We're not using groups support from the
Django LDAP extension and not doing so can cause errors trying to
fetch data from LDAP.
Huge thanks to @toaomatis for the initial implementation of this.
Fixes#72.
Previously we only did this when new human users were created via the
login process, which meant the management command to create a user did
not add the user to default streams (for example) and any future code
that might want to register a new Zulip user (such as the LDAP
integration) would need to import views/__init__.py in order to
properly set this up.
For reasons I don't understand, it appears that in Travis CI we're now
seeing errors using Casper that seem to correspond to a compatibility
issue introduced in PhantomJS 2, even though we're still using 1.9.8.
The solution for that compatability issue of patching casper's
bootstrap.js to get arguments from system.args at a slightly different
time than before seems to work in our setting as well, and that's what
this implements.
Probably the right long-term solution involves upgrading both
phantomjs and Casper to the latest versions.
This fixes a problem where the emoji_dump tool was not generating the
black-and-white emoji. The issue is that Pillow compiled without
libfreetype cannot extract those emoji (and gives an error of the form
"The _imagingft C module is not installed"), and if libfreetype-dev
isn't installed, pip will happily build and install Pillow without
libfreetype.
The do_send_missedmessage_events_reply_in_zulip function in the email
mirror didn't support EMAIL_GATEWAY_PATTERN that wasn't of the form
%s@example.com (which resulted in replies to missed message emails failing
to be parsed).
* Removes the hardcoding of an old version of Virtualbox (and doesn't
specify the version to avoid getting stale again over time).
* Flips around the langauge to assume you don't have Vagrant already.
* Makes clear that the first-time installation is a lot slower than
future runs will be.
Fixes#5.
Apparently it isn't supposed to work reliably with the container-based
infrastructure that we're using and empirically it's causing build
failures.
Thanks to @mijime for tracking this down.
This fixes issue #123. Namely, the script in scripts/setup/install was
returning 0. Adding `set -e` and `set -o pipeline` causes the install
script to exit and return 1 if any part fails, including piping output
(`set -o pipeline` does this).
In b59b5cac35, we upgraded our Google
Oauth code to support new python-requests, but because Ubuntu precise
still has old python-requests, this broke the codepath for older
systems.
Removes:
* Several unused <hr>s
Fixes:
* Odd linebreaks
* Inconsistent headers
* URLs which should be links
* Headers which should be headers
Code-formats:
* envvars
* FQDNs
* commands and command options
* config options
* code
This also requires updating the required version of oauthlib; previously an
appropriate version was being installed only because it was a dependency of
the wrong twitter library.
This only affects development environments and/or hand-built
installations relying on the contents of requirements.txt.
To fix existing environments, the incorrect api needs to be explicitly
removed with `pip uninstall twitter`.
Fixes#86.
Fixes a few major issues:
* Documents RAM requirements for running Zulip development
* Fixes missing steps in the "by hand" installation process
* Improves the emphasis in the section no how to run tests on the common case.
* Documents that you can use LXC on newer Ubuntu as well.
Most of our installation process is idempotent, but this step in
particular is not, so it's important to provide a clear error message
about how to proceed.
Ideally some of these templates should really point to the
local installation's support email address, but this is a
good start.
Exceptions:
* Where to report security incidents
* MIT Zephyr-related pages
* zulip.com terms and conditions
This is a bit hackish in that ideally we'd use proper options parsing
in provision.py, but it works and I even ran the tests 100x for tests
for flakes and didn't get any, so it's definitely an improvement!
With this we'll be both testing the runtime and effectively the Dev VM
setup process, which is awesome; the additional thing I'd want to add
tests for is the production setup process...
Previously, in Zulip voyager, the cron jobs would spew error emails
every time they ran, due to this directory not existing.
This also tightens the permissions for the folder and avoids needing
to create a nagios user for Zulip voyager; it should be writeable by
both root and the zulip user and world-readable (and thus readable by
the Nagios user on zulip.com systems).
Previously our redis config was built for precise.
Synced from redis-server 2:2.8.4-2 plus our one change, which is
disabling saving to disk, so just put that at the bottom for maximum
obviousness.
I wish there was a better way to represent the fact that this is all
we're doing, since this will make life more difficult for running on
precise as well.
Fixes#28.
requests 1.0 changed response.json attribute to response.json()
instancemethod. The code wasn't updated to match that change,
causing a TypeError when attempting to use the Google OAuth
Authenticator backend.
This is fixed simply by using response.json() instead of response.json.
While the docu on https://www.zulip.org/server.html says:
```
cd /root/zulip
./scripts/setup/install
```
This script downloads the `python-django-guardian_1.3-1~zulip4_all.deb` file to current working dir (`/root/zulip` if you follow the docu), but tries to install it from /root/.
This fails obviously. So i changed the download location to /tmp/.
This reverts commit 39f2908a32c0276b1d87ecedc876c71dd35a9b2f.
We're not including the preview_fail.png image in the release.
(imported from commit 2de1451de2f9b1727fc3a7e64c380b71c0f2caa8)
Instead, build them automatically when provision the development
environment and in update-prod-static.
(imported from commit aac8dfeaafbe872c113e5f2b6bd8f655a1af36f2)
This is in some ways a regression, but because we don't have
python-postmonkey packaged right now, this is required to make the
Zulip production installation process work on Trusty.
(imported from commit 539d253eb7fedc20bf02cc1f0674e9345beebf48)
This makes fetching settings from here for the rabbitmq config not
work; and the main place we check for this is in initialize-database
anyway.
(imported from commit fcc6eff3882f880fdc8b127daffc9a3173f3ff20)
This reverts commit b3aa3b44de198abea49f5b43e2403f466f30f66c.
The hook was actually being installed through the bind-mount into the
checkout outside the VM.
(imported from commit 7fcb4806743508e530b46593a1022ea00b74259e)
These features are in most cases possible to setup directly via our
GitHub services integration UI, and the customers aren't using Zulip
anymore, so this is worth doing to clean up the code.
(imported from commit 1e6f4ec523d85b6233a8e5b4eaa13eacfbe6e5f4)
One of the alert rows Was missing a "-status" at the end.
While we're fixing this, make it more robust by adding .expectOne().
(imported from commit 8cb0a560701e2ee48f78471ef8fe5cfa060782af)
It doesn't have any sensitive data since that lives in a separate
configuration file, and it's potentially useful.
(imported from commit 094e315439f8bd23ad07a8c2bc7d9776c8c7f096)
If there's a problem with Django settings then RMQPW would just be
empty, causing more confusing errors downstream.
(imported from commit 5948b1a15eb92fc032ea02e499be58365d8e9ecb)
The tarball build process runs in DEVELOPMENT mode, assuming it is run
on a dev VM (since then there is no /etc/zulip directory). Commit
d067bcfe9d71 made settings.py import local_settings_template.py in
DEVELOPMENT mode (then "not DEPLOYED"), not local_settings.py.
(imported from commit 9a08138d748dfca9c4ab8b366bee5c2fb96c25af)
Just importing zerver.lib.cache creates a file memcached_prefix that
is mode 0444, so we need to use -f or rm will prompt about whether to
remove it. Not sure why this is apparently a new issue.
(imported from commit 93c5140b66992339859e2b204c200d1dd7a35f2d)
We don't use it to set up the dev VM environment, but we do use it to
verify the Voyager puppet configuration parses in lint-all.
(imported from commit 3f116633ba3f535dfb4ae27915b6356e560ac84f)
These are the result of either the upgrade to Django 1.8 itself
(username max length increased to 254), or the changes needed for
Django 1.8 compatibility.
(imported from commit 6b1d7e73c85e9a2f7de9e5b91d851977eb4959e8)
This also removes the convenient way to run statsd in the Dev VM,
because we don't anticipate anyone doing that. It's just 2 lines of
config to configure it anyway:
STATSD_HOST = 'localhost'
STATSD_PREFIX = 'user'
(imported from commit 5b09422ee0e956bc7f336dd1e575634380b8bfa2)
The tests run as iago, who is now an administrator and therefore has
control over many more bots. Be specific about which bot to operate on.
(imported from commit 7a9d3e12da905338624747dd402702bb66907cfd)
django commit 596564e80808 stores the user id in the session as a
string, which broke our code that extracts the user id and compares
it to the id of a UserProfile object.
(imported from commit 99defd7fea96553550fa19e0b2f3e91a1baac123)
This commit loses some indexes, unique constraints etc. that were
manually added by the old migrations. I plan to add them to a new
migration in a subsequent commit.
(imported from commit 4bcbf06080a7ad94788ac368385eac34b54623ce)
We can add it back later but for now we can just stick with localhost
since that's what most people will want.
(imported from commit c5fe524282219dc62a0670f569c0cb6af04be339)
Fixes
[
File "/srv/zulip/zerver/lib/actions.py", line 605, in recipient_for_emails
if not (normalized_emails & admin_realm_admin_emails or normalized_emails & settings.CROSS_REALM_BOT_EMAILS):
TypeError: unsupported operand type(s) for &: 'set' and 'list'
(imported from commit f39a95dad7b3207e9188fc03926cd116061ef3f3)
We don't need to check whether the user exists before creating it:
CREATE USER failing is fine.
(imported from commit e8b2bc5495e328ee30d15445a566c0edff2f069d)
If we run provision.py a second time, there will already be
zulip/zulip_test users, so the CREATE USER will fail and the password
won't get updated to the newly generated value. By creating the user
and setting the password in two commands, we allow the creation to
fail without affecting whether the password is set.
Also the quoting for updating .pgpass was wrong.
(imported from commit 5e249813c17cb4829e4e4958e92aaa30563c5f96)
Sometimes I get the error "Selected message id not in MessageList"
when running the casper tests. I think it's probably when the test
user's home view does not contain any messages.
Ideally we would fix this in a way that guarantees that we generate
whatever messages the test suite needs...
(imported from commit 51a02da612dda88d60681b9e09cd6e6a2c39a470)
Create a .bash_profile in the VM that activates the python venv on each login.
Node dependencies include some npm packages that haven't been backported to trusty yet.
(imported from commit 8307740004a05b0c9d6ea26f97c2e80b9d1bbaf4)
Source LOCAL_DATABASE_PASSWORD and INITIAL_PASSWORD_SALT from the secrets file.
Fix the creation of pgpass file.
Tim's note: This will definitely break the original purpose of the
tool but it should be pretty easy to add that back as an option.
(imported from commit 8ab31ea2b7cbc80a4ad2e843a2529313fad8f5cf)
Include new field on Realm to control whether e-mail invitations are required
separately from whether the e-mail domain must match.
Allow control of these fields from admin panel.
Update logic in registration page to use these fields.
(imported from commit edc7f0a4c43b57361d9349e258ad4f217b426f88)
We still need to put in the work to support other platforms, but now at least
we'll error out if you're not on 64-bit Ubuntu 14.04.
(imported from commit 3a35953206906044947e3447c7ab8fca78a76e1e)
The old language was confusing because "the interface" could refer to something
like eth0, but in actuality refers to the IP/hostname to listen on.
(imported from commit 4f77d72a4dfcdbe7e7747c6228975aa68dfbe6ac)
Meant to be used in tandem with the manage.py import command.
The following sensitive data is scrubbed:
* user api keys
* user password hashes
* stream email keys
* invite-only streams
* messages from invite-only streams
* messages from users from other domains
(imported from commit 8e58dcdcb80ef1c7127d3ab15accf40c6187633f)
If you're using e.g. our Jabber<=>Zulip mirroring capability along
with the RemoteUser SSO integration, previously it would fail if a
user with a corresponding dummy user tried to login/signup (since they
didn't have an account but one wouldn't be created because
ZulipRemoteUserBackend was reporting that an account already existed).
(imported from commit 006eaa9afa8feedddd860c2bef41e604285228a7)
We also reference these secrets from zproject/local_settings.py, keying
off IS_DEPLOYED.
(imported from commit eb83310e219616ed1c6c253f0d6893134bbe3517)
It's been very buggy for a while, has limited usefulness compared with
unread counts, and profiling over the weekend indicates that it's very
slow.
(imported from commit 716fe47f2bbec1bd8a6e4d265ded5c64efe2ad5c)
Also increase the number of messages sent as context from 5 to 10 and
look up to 15 minutes in to the past for context.
(imported from commit bfaed9bcff1ee2047fc3b7a63acf93cd2d47cc7d)
Now we have 2 different Zulip apps out there, and they are signed with
two certs: Zulip and Dropbox. The Dropbox-signed apps are going to need
to be sent APNS notifications from the appropriate APNS connection
(imported from commit 6db50c5811847db4f08e5c997c7bbb4b46cfc462)
Pages from MP are using the description field not the subject field.
Include both in the page if given and don't fail if the key is missing.
(imported from commit 4351e5656d4ea025a03c07c8bb3bb5d406ef2d3d)
The feedback bot needs to be included in the OG_ZULIPER_EMAILS so users
can send feedback.
(imported from commit b0c3295379d0f062a912f8b6c957419335be1c48)
The SSO flow which was never used on a realm with mirror dummies before.
Also change the redirect to stay on the same doain.
(imported from commit 0f1b8a8fcef82ae6eaa5a264686f98d62a683fac)
This commit should only be pushed to stage after c290b630e has been
pushed to prod otherwise it will create a redirect loop.
(imported from commit 408407b845ded596705b1abd8ad13c0aedf6d732)
We were trying to default the users first name when using google auth,
but it was getting lost when rendering the form.
(imported from commit 710e0c2ce591488920458dca74209c75e7031abd)
This change disabled password auth, but the UI still shows the login
form. I will remove that once we have the new hostname.
(imported from commit 6ca119571854ac54645680b40255e346be1c1613)
This needs to be deployed on both prod and lb0 to be functional
DEPLOY INSTRUCTIONS: restart carefully
(imported from commit d97a450754608357418c80e5b3c7b3bbcd1d09fb)
This is safe because we have the wildcard-all cert.
DEPLOY INSTRUCTIONS: Change the CNAME in R53 for external-content.zulipcdn.net
to the same as www.zulip.com
(imported from commit 075984943ce3a3b17518b913ea650992e45f705e)
This change will redirect armooo@dropbox.com from stage to prod. It also
removes the prod to stage redirect for all users. This will be rolled
out in two commits to prevent a redirect loop.
(imported from commit c290b630e746f757429b8bbdadbe7768367a5e33)
Resolves a AmbiguousTimeError.
Approved by Leo.
This reverts commit ebfaeb97ffda22b618be7a9206877f9d2ec53404.
(imported from commit 42b29c6c57eb954952a740bc89611031cef1834a)
We were serving 401s on /user_uploads when the user wasn't authenticated (due to
it being a REST endpoint). This was causing a login popup to display instead of
just a broken image preview.
(imported from commit 62640f5bd59eb3b86ab5aae5923ccfa742459805)
I could not find where we were setting the read flag on messages in
response to a update_message_flags event. This fixes a bug where a
user's read position will not be correctly synced in muted streams. For
muted streams the cursor updates seem to force the client to mark the
messages as read.
(imported from commit e7e392be4c8cbf6f734abfa7fee748b07fd495bb)
This is a public mirroring bot that needs to be able to send forged
messages to a stream.
(imported from commit 3fa691b1f1d06bf68a8cbc2c31ed5e3e5efef177)
Missed message email were including the context messages in the number
of messages you were mentioned in.
(imported from commit 1749c5d272d2e17d6e28456ace932f80715103a3)
* Fixes a few bugs with missed message address for PMs and huddles.
* Uses missed message address for all missed message reply-to headers on
the zulip.com realm.
(imported from commit 61dd09386e1bbdf9a5096e2400984d31e73a5b74)
The one time use email addresses are prefixed with mm and need be sent
to the local zulip user to be picked up by the email mirror.
(imported from commit e17cfe6855ab7886f25ded52790b8f31df955ef2)
The one time use address are a unique token which maps to stored stated
in redis. We store the user_id, recipient_id, and subject. When an email
is received at this address it is sent to the stored recipient by the
stored user. Anyone with this address can send a single message as this
user.
(imported from commit 4219417bdc30c033a6cf7a0c7c0939f7d0308144)
Send a different missed message email for each recipient. This allows us
to set a different reply to address for each one. PMs and huddles use
the existing logic, replies will be sent to all parties via email.
Missed @-mention emails will have the reply to address set to the
stream's email address.
(imported from commit bfb7cf7c1382adbf3720caa74cbb927c10dea267)
We were expecting Github to send us the string "true" when the exclude_* options
were set. However, we were actually getting "1" when an option was set and the
empty string when unset. So we were always setting the options to False.
(imported from commit 067ba60b0b0404aebc6eda9487b1201fc2764243)
One common place that this happens (for us) is on a local
Dropbox .dev.corp.dropbox.com instance, which can't be reached
by the Zulip servers.
This commit also:
* Fixes the test suite
* Properly previews /photos/ links
(imported from commit b4788b6236e7a9d390e1efc4673be34d9ba5e091)
After I reverted the change to the bot stetings page, this
broke a test. This commit fixes that.
(imported from commit 394b29fea4f75096f7cb8d819145a9adc386276b)
--jabber-server-address=JABBER_SERVER_ADDRESS
The hostname of your Jabber server
--jabber-server-port=JABBER_SERVER_PORT
The port of your Jabber server
(imported from commit f6b73ce6a2e3dcdbb100a1b6aba5ad89fc99a457)
This can be used by mirroring scripts to only forward messages to users
who do not zulip accounts.
(imported from commit 200d6bcaaf39238bfb01480a9e906d567d4d9e11)
Truthfully, the actual way to do this is going to be a bit
more involved and also involves changing Realm.NOTIFICATION_STREAM_NAME,
probably on a realm-by-realm basis.
(imported from commit b6a05849d215e07ee6716d116ff5e2c819d5b4be)
Known issues:
* No support for whitelabeling in the email
* No whitelabeling for any externally-visible branding
(imported from commit 9eab7b0744e56a87007b8621a8bb18bbb1080256)
When you are at mentioned in a stream we will now send you up to the
last five messages which were sent in the past 5 minutes on the same
topic and stream.
(imported from commit 6df6c1cf868722a7bf76e54710e38741a7ac8f31)
When an event queue expires the client is in an unknown state and trying
to restore state during a reload will keep the incorrect state.
(imported from commit e0828626142029aecd86a7c4cec8c77d261eb3eb)
Activation emails were using django's sites framework which always has
the domain set to zulip.com.
(imported from commit b81eae96e1a75b64dd93970760b869f3271ce88c)
The default today is to not have issues traffic except on a whitelist. This is despite the fact that we have
an exclude_issues boolean on Github's Zulip-integration page, since if we changed the default, all realms
currently using this default would have to go make this change on every repo. That's something that would require
some work, in terms of communicating with them about this, and logging integrations settings for all realms, to
see which are correctly setting exclude issues. Unfortunately this probably isn't high priority today, but let's
try to get this whitelist change out to prod ASAP.
(imported from commit 256fe32bb6aaf7de18ff02d8d7e204a13bc02b7a)
Chrome has removed the webkitNotifications API and not only has the w3c
web notifications API. This adds a shim when webkitNotifications is
missing but Notification is present.
(imported from commit e21c476f9ae6570c297c88bd6ff90a97818688e6)
Display a red warning box to get users to direct users to staging for
the zulip.com (dropbox) realm.
(imported from commit 01ad4209d9247406bc82f5dedaf21371101a1d84)
Apply this commit after hours!
To apply this commit, first run the migration and then run the following as the
zulip user on staging:
$ echo 'VACUUM zerver_message' | python manage.py dbshell
The above VACUUM is needed to clean out the existing fast update pending list.
It might take a long time and block new message inserts!
See discussion near Zulip message 18377486 for why we're turning off the fast
update mechanism for zephyr_message_search_tsvector.
The high level overview is:
As a consequence of the high work_mem setting on our postgres server, the
fastupdate pending list for zephyr_message_search_tsvector can grow very large.
This leads to the occasional INSERT or UPDATE taking inordinately long (many
minutes) as the pending list is flushed, blocking other inserts.
One other possible solution for preventing the list from growing too large is to
set the autovacuum storage parameters on the table such that the autovacuum
process will run after a reasonable number of INSERTs or UPDATEs. However, the
table is mostly INSERT-only. Therefore, only the autovacuum_analyze_*
parameters will actually do anything to affect when the autovacuumer will run,
but when it does, it will do a VACUUM ANALYZE instead of a plain VACUUM. We
don't particularly need the table to be re-analyzed that often.
Turning off fast update will eventually cause the index to become less
efficient, but we can always rebuild it later if we notice it starting to get
too slow.
(imported from commit f280c193c3bc0a3f312960510c5a7dcf97f30c3d)
This can happen if the calling process is handling SIGCHLD. See
http://bugs.python.org/issue9127
We ran into this in the zephyr_mirror.
(imported from commit 80fade2274714b7c2c4b9fe38c66a1db8cc63234)
andersk reports that we are "incorrectly sending outgoing zephyrs with z_charset
= ZCHARSET_ISO_8859_1, which breaks zwgc (other clients tend to ignore
z_charset). I'm assuming you're shelling out to zwrite; you need to use zwrite
-x UTF-8, or export LC_ALL=en_US.UTF-8."
(imported from commit 8abb28bf8acbe0b5aa99ca13faab24e1e554a031)
add_messages is a good entrypoint for this, since it gets called by:
1) get_old_messages
2) get_events_success (for new messages, via insert_new_messages)
which is all the places that rewrites should happen, but nowhere
where extra work is being done.
(imported from commit 844c33bc32d35aa39c9cdacf42eb7e8ddf5ae63c)
URLs with a realm of "unk" will be queried against the new bucket to
determine the relevant realm of the uploading user.
(imported from commit 5d39801951face3cc33c46a61246ba434862a808)
Thanks Tom Cook for getting these through Digicert!
We no longer need separate wildcard certificates, etc, because we have SAN star
certs.
(imported from commit 40a8961da51b6a0ae90c68b40b2af6d59cb5cf9f)
Both missed in the refactoring that split global messages and
message_list_view message_containers.
(imported from commit 127d09204a9e363b78eccfe3d72212e78beb2600)
The idea here is that for usages like in the zephyr mirror bot:
backoff = RandomExponentialBackoff()
while backoff.keep_going():
print "Starting zephyr mirroring bot"
try:
subprocess.call(args)
except:
traceback.print_exc()
backoff.fail()
we want it to be the case that the mirror bot running for a while counts as a
success so that the bot doesn't have a finite number of crashes over its entire
lifetime. We only want the mirror bot to stop retrying if it fails too many
times in a row.
(imported from commit 7b10704d3ce9a5ffb3472cbb4dfa168c9c05ae7a)
Collapsing a message in a narrow should also collapse that message in the
home view. Previously this would only happed with the message was
rerendered.
(imported from commit fa82888eba51eb2f4f2b93521d4b7daee852898d)
This allows us to specify different rules for the zmirror machines, which need
ports open for Zephyr.
(imported from commit f3c061e9492cbb99783f156debccf03161347e47)
This removes "X-Frame-Options DENY" from our nginx config. We need to be able
to load Zulip in an iframe for embedding and we decided that it doesn't actually
provide much protection.
(imported from commit 5bc363693db949010f6163cb3000c12229618a83)
Otherwise the user_profile.backend attribute doesn't get set. I didn't notice
this previously because on first register authenticate() gets called, and then
the UserProfile object gets cached. This means that subsequent logins work just
fine as long as the UserProfile object is in memcached.
(imported from commit 834d95c46aa07724ea84802f09b7249de99b5ca8)
CUSTOMER16 wants their employee realm to:
* only use JWT logins
* have name changes be disabled (they want users' full names to be the
their CUSTOMER16 user name).
* not show the suggestion that users download the desktop app
(imported from commit cb5f72c993ddc26132ce50165bb68c3000276de0)
We currently expect the use of HMAC SHA-256, although there shouldn't be
anything preventing us from using other algorithms.
(imported from commit 354510a0b7e9e273d062a1ab5b2b03d4a749d6a3)
When the date changes between an existing group and a new group the
existing date separator needs to be updated. This is done by rerendering
the existing group.
(imported from commit a3775815e33872b0ec07704dc7ccf5fd2671fa21)
update_rendered_message_groups needs to use the message not the
message_container when testing to see if the fade states need to be
updated.
(imported from commit b1c3baba07169a369d827c89afdc3c406ada0b79)
Now that we are not directly using message in the message list view
rename the uses of message that are message_containers.
(imported from commit 5c355703a8934a74864f5de6ecb1e2fd851e5d41)
The messages being passed to the handlebars templates were global
messages which we were adding per list details to, show name bar etc.
This causes rendering bugs when you try to rerender a message, because a
different list may have changed it. This commit moves the global message
data to a msg attribute on the message_container which will contain the
per list attributes.
(imported from commit 26b1f0d2c72d6288a6d3e7ed5f8692426f2a97ad)
When clear_table is called message_groups must also be cleared.
Otherwise render will try to incrementally update the DOM which will
fail when the expected existing nodes are not found.
(imported from commit 5ec3ce01717741b17c719fabded316619cdc4b25)
The handlebars template adds a text node with a newline after every
message. So we need to filter the jQuery object to include only the
message rows.
(imported from commit 07513b485e805570e450fb93c07091be89bcbd50)
This way if two browsers are disagreeing about your active status, the
active one wins. The active browser continues to update your timestamp,
and the idle browser's changes are discarded until the timestamp on your
active status expires.
(imported from commit dc29e013d045c4b72793097f611ba6802c58e57a)
We apparently still have some process that occationally sits idle in a
transaction for a while, which makes this alert super noisy.
(imported from commit 074b04ad746bac0da1b8714763538d1ce22da64e)
Passing anything other than an array of DOM elements to
_post_process_messages is an error. In this case we were passing an
array of arrays of DOM elements.
(imported from commit 9e3be18598c406f3578a867dab36731ffeeac921)
The goal is to have a more data centric piece that can be unit tested.
We also try to minimise the number of one off jQuery DOM updates and
rerender handlebars fragments instead. This will prevent the
message_group and DOM from drifting apart and not being able to rerender
correctly.
(imported from commit 03f09803f2bc0c3b8187f76f2cfe90be9f7512a3)
To make the rendering process a bit simpler to read this commit is
refactoring the message group creation into its own function.
(imported from commit b53ce96ed8fee3064d7cf891fc248d0c3d821d1a)
We are seeing error on CUSTOMER4 when clearing the DOM on reload. So
now we will only clear the message list.
(imported from commit f5d8d7d36cd1018f7def73ff9eda414387fcec5c)
Previously, you'd have to be offline to recieve missedmessage
notifications, or maybe idle for an hour. However, I'm pretty sure the
latter code didn't actually work, so we scrap that and just nofity you
via email or push as soon as you're idle.
Closes trac #2350
(imported from commit 899966e0514db575b9640a96865639201824b579)
I don't think this matters a ton, but there's no reason for
the two muting files to come before setup.js.
(imported from commit c18231b633fba6acccaf5bc180c7605f7591462e)
Since it's basically impossible to add a person with an
undefined full_name--even "skeleton" people--there is no
need to check the full_name field to short circuit reify(),
because it will always be defined.
(imported from commit 3a30cfd583a040f7460739abea1604594c450ffe)
Doing so requires superuser privileges because check_postgres.pl only connects
to one database for that action. We could theoretically work around this, but I
don't think it's worthwhile for non-production DBs.
(imported from commit 3ab06e4dd6f844c81128b81709cdc3cdfbe37c47)
We believe these will generally no longer be disruptive now that we have
autocommit enabled.
(imported from commit c8c1301e0d4b188d6708173cd8c8b16279e3d910)
Closing the edit box earlier will make future changes less brittle,
when we, for example, re-narrow based on topic edits.
(imported from commit 36219c5129153beebfefe443932825fdf74abc43)
This helps the edit form in particular, when you change a
topic and need to select the propagation option.
(imported from commit c9dd1e62cd9e0b2142855685f04baa06eecf7226)
If we get a topic change, we can change the subject outside the
loop, since we are passed in event.orig_subject. Doing it inside
the loop was mostly harmless, since after you encountered the first
message with the old topic, the condition to change the subject
evaluated to false, but it was still technically O(N), and it was
kind of confusing.
This commit changes behavior in the edge case that you have the
compose box open for a changing subject, but you are in a narrow
that does not have any of the affected messages. After this commit,
the topic in the compose box will still change, which I believe
is the correct behavior.
(imported from commit 2363e432ebe7ae8e07379324ee0bfb52051428e6)
Before this change, we were incorrectly trying to do local
filtering on negated has searches.
(imported from commit d1a6f1feef6b3cc1c984eb91a73cd16c4e66874e)
We use this error page for all 5xx errors, so I'm removing the error
number from the text.
(imported from commit c0c4c8829d466dd20c76dbbc920fc57cafbd6e6c)
We still don't show this in the frontend, aside from our usual "Not
delivered" message that we also show when you send to a non-existent
user.
Addresses #2349
(imported from commit 2f348b15a4d539987ddbcccbbf40e2be87c1f92d)
We show a user as "on mobile" if:
* They are only active on mobile
* They are inactive on all devices and can receive push notifications
(imported from commit 0510b9371727cd19c72f6990df7112921c36ad48)
This doesn't affect code when not in testing. It shaves 7 seconds off of casper
test time on my machine.
(imported from commit 7e27fa781bcf16f36d9c8f058427ba57c41068bd)
Normally, casper delays checking the waitFor condition for 100 milliseconds and
further does not act on that check for another 100 milliseconds. This is just
silly.
(imported from commit ad046ceda81abda5c609ce25ef0d4fb27d3da716)
send_message -> then_send_message
send_many -> then_send_many
wait_and_send -> then_wait_and_send
Hopefully this makes it clearer that they should not be called inside of steps.
(imported from commit 4fcc971817b25056100311ba55303da2c5527f0f)
Casper was calling casper.then(then) instead of calling the callback directly.
This meant that the callback was being added as a step, which worked, but was
not consistent with the rest of the casper model.
(imported from commit b3bf916f7c56dd3d4e7be3569ebdf9d3045cd085)
This speeds up CasperJS tests by 25 seconds per main app page load.
When we switched the SockJS, the casper tests got inexplicably slower. I
finially figured out what's going on. The first SockJS XHR request (remember
that we don't get websockets in the test suite) gets considered part of the page
load and therefore the PhantomJS onLoadFinished handler doesn't get called until
the SockJS XHR finishes, which happens at the heartbeat, 25 seconds later. To
fix this, we simply don't create the SockJS object on page load since it will be
created on demand, anyway.
(imported from commit 845a97526c5102df426cd6fc26182a734e7fcab6)
Catch any exceptions that happen in the process of triggering
the message_rendered.zulip event. This addresses #2356.
(imported from commit ce771483cd2533d312fbd68e9c2753c80b3c8d49)
Our restructuring of the messages (especially grouping) seems to be the culprit for message copy and paste
(imported from commit 14632a67f55efea4f1b53cc718a4f655ac83b387)
This will make it slightly easier to consume the data from our clients.
Ref:
RFC 6585 §4
(imported from commit 6d323dc25db78a6d84a163add950f039e03e73d3)
This addresses #2351. While I could see the argument for
wanting to edit a message without changing your selection,
I think it's just very surprising behavior and inconsistent
with the rest of the UI.
(imported from commit 3bb4faca0656258b76bfaafbd7f4a645810578f6)
In a test run with a hand-constructed query, this sped up the query time from
280ms to 50ms.
(imported from commit 8cbe199ca50a487491d13d6d6ef940ea668c1038)
We were previously calculating it from the zulip_email option, which might not
be set if it is being specified via the config file.
(imported from commit 76866c239ca63ef90a7967c9a6027aeec9be6390)
We can't just check that the realms are the same because ist.mit.edu is an open
realm and uses @mit.edu email addresses.
(imported from commit 7dbaa81cea6e4f82563dfc0cfe67a61fe9378911)
See #2357. We now support `~~~ .py ` with that trailing space.
Note that the test coverage is Python-side only due to
bugdown_matches_marked being set to false, since we don't yet
support language syntax on the client side.
(imported from commit ccd5fcb0eee01478d349161400103480678d7486)
Previously, if you searched for "in:home search:foo", we
weren't making "in:home" a public operator, so the back end
wouldn't know to exclude muted messages, but the front end
also wouldn't exclude muted messages, because it assumed
that queries with "search:" in them were fully narrowed by
the back end.
Prior commits made it so that the back end is now capable
of doing "in:home" narrowing, so to get the properly narrowed
results, we simply needed to make in:home be a public operator
in this commit. We also made in:all be public for convenience,
although it's essentially a no-op.
(imported from commit e4a8b10813b50163c431b1721bd316b676be1b83)
Adds APIs edit a bot's default_to_stream, default_events_register_stream
and default_all_public_streams.
(imported from commit c848a94b7932311143dad770c901d6688c936b6d)
Support setting default_to_stream, default_events_register_stream, and
default_all_public_streams during in the bot creation API.
(imported from commit bef484dd8be9f8aacd65a959594075aea8bdf271)
Allow bot owners to set which streams their will receive events for
without needing to change a configuration file.
(imported from commit 2b69e519dbc12ffbdba072031a7f7196c9e50e33)
This allows bot owners to configure which streams messages are delivered
to without needing to change webhook URLs or configuration files.
(imported from commit 32a0c26657c145b001cd8cb3ce0a0364d48902ce)
This commit finishes up support for has:* searches by adding
the front-end pieces, specifically the part that "has" operators
will not be applied locally. It also implements basic
descriptions for search suggestions and canonicalization
of operands from plural to singular.
(imported from commit a3285bc33d06d76b5a2b403ebcdd911b4cc03980)
We don't have room in the dialog to show every variation of
the negated search syntax. If we start by documenting negated
topic searches, we may get some discovery on the negated search
features without cluttering out the way more common search
options.
(imported from commit 3b2c2672d125788dc71a5eeb2b6f1e551380c62e)
This fixes#730, which includes any customers we want to notify
about the feature being pushed.
(imported from commit c60959ddd7c61ea8e014e984047e9f7bc0d59296)
Typing "stream:foo -topic:b" leads to "stream:foo -topic:bar" properly
as a suggestion now.
(imported from commit bb0acf52744f7b13977a3db5d3c130d1402b09b7)
This is needed for the ist.mit.edu realm, where the same email address could be
used for all three of Zulip, Jabber, and Zephyr. To disambiguate Jabber mirror
dummy users from Zephyr mirror dummy users with the same email address (but
which would be in a different realm), we force all email addresses in the
ist.mit.edu realm to have local parts that end in "+ist".
(imported from commit 5ebad2cc8f6c7286b4b9426d57ed75578703fb86)
It doesn't actually seem to be necessary to adjust the SSL version in current
versions of Openfire.
(imported from commit 6766d9fd5081cf16f2a5e3e2b008f20e7498caa4)
We also take the opportunity to use the sleekxmpp JID parsing more instead of
doing string manipulation.
(imported from commit 6e4ba0bd2c241666fcde42333ff68b879d8ab2b7)
This doesn't change the alerting UI logic, it just turns
alert_words_ui into a module and calls the setup code from settings.js
when the settings page is rendered.
(imported from commit 05f95383b046086641280f82f648be58688efe61)
We should hide the empty narrow message message whenever the narrow is
not empty, but we should only select a message if non are currently
selected.
(imported from commit 22f6b0827dc84ed587a83b5d713b12c1d5d4c0a0)
Github flags pushes as either `forced` or not. However, it always marks new branches as
forced pushes--but we don't necessarily agree with them. This commit checks for the `created`
flag as well.
This resolves Trac #2346
(imported from commit 960bd3ad707a4d1ad431e21dcd79389e8d4b297b)
The match_subject and match_content template vars are notorious
for causing bugs due to the way handlebars forces the strange
../../.. syntax on us, so now we have some test coverage.
(imported from commit c6b151b964ae8b6fb199d9cdbe533a87c6b58947)
Testing directly against NarrowBuilder is convenient, as it
requires very minimal data setup to get a basic sanity check
of the SQL that gets generated.
(imported from commit 5f3bb0364713bd2e4228a9b9d4d16bde297b4e16)
Before saving a Message object, call update_calculated_fields()
to set the has_attachment/has_image/has_link fields.
Note that the pre_save hook we added here does not get called
if you call bulk_create, hence the explicit call to
update_calculated_fields() in do_send_messages().
(imported from commit 1d60ae5908ef186aa5ff1e39277dbb2b765e60d4)
When reloading the page we want to narrow to the location that is
restored from the query parameters. This is only done if we ask narrow
to use the first unread message from the server.
(imported from commit b585ef51cbb85788b24d90d831b42c45fd188569)
The reload initiation is required to run before other parts of the site
are started so that page_params will be setup correctly. This moves that
initiation out of an on ready handler to an explicit initialize call
near where the rest of the app is started.
(imported from commit b8994311299327aa3cfa57e3d9e92124a47123f4)
rerender_messages() does extra work such as making rerendering in narrows
safe, as well as updating recipient bars. That should be the only valid entry
point for rerendering individual messages
(imported from commit f91aeb2070b1056ab95e01d68a342558c2813ae8)
Previously, we were using U+1B80, which is 'SUNDANESE SIGN PANYECEK' (i.e., an
assigned character). U+1FFFE is a non-character (i.e., free for internal use)
on Plane 1.
(imported from commit f93d2401733afb96b6d717a7f659c011299071df)
For personal mirrors, this means that the mirror should join all Jabber rooms
corresponding to their Zulip subscriptions. For the public mirror, this means
that the mirror should join all Jabber rooms corresponding to Zulip streams that
end in "/xmpp".
(imported from commit 16ed9e3deee7e02b2ae0dd79925f0ed5c4acd044)
A stream is vacant when it has no subscribers and occupied when it has at least
one subscriber.
We have a slightly odd model where stream creation is conflated with
subscription creation. Streams are created by attempting to subscribe to a
stream that doesn't exist. We also hide streams with no subscribers from users
to make it seem like they've gone away. However, we can't actually remove those
streams because we want to preserve history.
This commit moves us towards a separation of these two concepts. By sending
events for stream creation, occupation, vacancy, and deletion, we allow clients
to directly observe the global state of streams rather than indirectly observing
subscription information. A more complete solution would involve adding a view
for explicitly creating streams without subscribing to them.
This commit does not handle the intricacies of invite-only streams. We
currently simply do not send these events for invite-only streams.
(imported from commit 5430e5a5eecefafcdba4f5d4f9aa665556fcc559)
This includes removing GET support for the endpoint, which is unused
and doesn't map well to this being a bulk endpoint.
(imported from commit 348ff9dfa84be1661368c6d7d35aebf2ae2a9ae0)
Previously, streams used `notifications`. Now they have separate
`desktop_notifications` and `audible_notifications`.
On staging, this should be run after the schema migration has been
applied (and technically before the code that uses the new fields is
deployed, but for staging, processing our notification settings a few
minutes late won't hurt anyone).
On prod, the script should be re-run just before the code using the
new notification settings is deployed to process any customer
notification settings changed since the staging deploy.
(imported from commit d99d238cd1b317c5180d7f940d70a7e2f8f9c712)
`/usr/bin/env python` is almost always preferred over specifying the
specific python to run (and this script doesn't work for me on OSX
with /usr/bin/python specified).
(imported from commit 531e6062ba0ac1f25e3c681bb5cf83a918d0e3e7)
This helps the common case of not liking our default of having audible
and desktop notifications enabled, and not making users adjust the
settings on every existing stream to fix it.
(imported from commit be75edb2c1385d1bd9a289416e2dffd8007f5e0a)
They have weird properties like not sending anything for unchecked
boxes, which makes it hard to wrap a client-agnostic API around.
(imported from commit fef73a57a55b218b55dab6be3453dd6eac73c789)
This migration will do nothing on staging/prod since the indices already exist.
It is only for creating the indices in dev.
(imported from commit ac26a23641191ba73fbccc2eebc4a261ece6c624)
We will need to run these commands manually when deploying to staging:
CREATE INDEX CONCURRENTLY "zerver_message_has_attachment" ON "zerver_message" ("has_attachment");
CREATE INDEX CONCURRENTLY "zerver_message_has_image" ON "zerver_message" ("has_image");
CREATE INDEX CONCURRENTLY "zerver_message_has_link" ON "zerver_message" ("has_link");
(imported from commit 84808dc6b1af887ddf784cb8a875ae462f4df985)
`#tab_bar_underpadding` overlaps some with `.message_header`,
so adding `#tab_bar_underpadding.bottom + .message_header.height`
gave us a nonsense message viewport top.
Doing the calculation this way is more robust, as long as:
1) `$(".floating_recipient").offset().top` continues to give us a sensical number
and is the last element just before the top of the viewport.
2) nothing appears between the composebox and viewport.
In this commit I also removed the other couple of places where the #tab_bar_underpadding
was being used as a viewport reference, that no longer makes sense.
(imported from commit c7f35e41309900c581d5e2329c1becf161d501d3)
This commit makes it so that we don't individually query
auth_permission for every user on the realm during calls
to Realm.get_admin_users(). This should speed up page loads.
To apply this commit, we had to upgrade all of our servers
with this patch to django-guardian:
https://github.com/lukaszb/django-guardian/pull/178#issuecomment-31049062
(imported from commit a1604bf573a5005c9abc128a680a7da6a20cabef)
Before this change, the way we'd strip tags of punctuation
was just sort of messed up, because we'd strip the start tags
one way and strip the end tags another, and we had conditionals
for the different flavors of tags, instead of doing the stripping
when we already knew what flavor of tag we were dealing with.
(imported from commit 60c5ebd45e21b88bbfc98ff4b43dbbc6b32b38a1)
This allows clients to show an indication if users are on mobile or can
be reached via mobile.
(imported from commit 819c27a141f71dee32a97fd5c1ac405b039a5038)
Also rename the function, which was previously called to_presense_dict.
This will let us be more flexible about adding additional values.
(imported from commit 323b7d2df53918928190c9ee9544b4599a5e2df3)
If we call exclude_muting_conditions() with a non-stream
narrow, it will now include a condition to exclude streams
that are not in your home view. As of now, this code only
executes during testing, but it sets the stage for doing
better in:home queries on the back end.
(imported from commit bbd764bd0e9588a50e4a82c915e82a2c1b99d73e)
If we are already narrowing to a stream, then we can disregard
muted topics in all the other streams and create a simpler query
for the DB to execute.
(imported from commit 35a074a76eec99922034a381741355da3fdd5b39)
Due to the way we store muted topics, it is possible that a
muted topic stream name may no longer exist, and we need to
handle that case gracefully.
(imported from commit 4d18ec55e45213657a67e160848229678f212765)
Previously, the email mirror queue worker used the API bindings to send
messages to Zulip, as if it were any other API client.
This is inefficient since we're running the worker inside the Django
context on a machine with database access; we can instead just use the
internal message-sending functions we use elsewhere. This also resolves
potential issues with SSL certificates, etc. that might occur when we
were previously making a HTTPS connection.
(imported from commit 6de8015829bec440f1af0199a2138828e86ed2a4)
Previously topics weren't being highlighted at all and messages had their
highlighting persist across different narrows (because we were only checking
whether the message object had a match_content property, not whether it should
currently be used).
(imported from commit 44c91c6d5799dcdf765e19e1a17bd727ce80c918)
Previously, digest emails provided links to Zulip that didn't correctly
encode "/" if it occurred in a stream name or topic. By explicitly
specifying «safe=""», we can request that urllib.quote escape such
slashes.
Closes trac #2294.
(imported from commit 2e6334672969d4cf4032d2ea5dc80091af96d672)
This requires a puppet apply on prod, as well as manually
updating the symlinks of Zulip-latest and Humbug-latest on
prod0
(imported from commit c5ef8cd0e2d156144531b35af9a8c5226f5bf750)
As part of this, I also made test_basics() have a third
stream that makes false positives in the test less likely.
(imported from commit d5ba64ec9346741818e30abe9e9594788c339fab)
When navigating away get_events will fail after we delete the event
queue before leaving the page. In that case we try to reload the page to
correct the problem. This happens before the page navigation is
complete, and then we reload zulip keeping the user captive. This was
only observed on on Firefox.
(imported from commit e001172e87a9f2ab7cf07a477e46b9d87752ac04)
Now that we no longer use tables for our message list, we can
more logically group messages together.
(imported from commit 9923a092f91a45fe3ef06f2f00e23e4e3fb62a37)
This experiment has been disabled for everyone for a while: if we
bring something like this back, it is not likely to be exactly the same,
and will be different enough to require a different implementation.
As it is, the summarization code was making a few code paths (rendering
especially) more complex, and is worth removing for simplicity's sake.
(imported from commit 6ac8cdc9f7077a5a1da01ab4268aba3db0bc43f8)
If you have a lot more individual users in your realm than group
PMs in your recent history, we were squeezing out the Group PMS.
We now max out the ratio for any sub-section in the right sidebar,
as well as bumping up the min.
(imported from commit f7c44367f2a518d27406993cb6358cc96d1aae92)
Previously, we assumed that num_before or num_after would be always be non-zero
after adjustment for the anchor. However, we don't adjust num_before or
num_after when a narrow is specified.
(imported from commit 9239fef140e109b11bdfbeef42e9fbed78660ad1)
This changes Filter.describe and Filter.operator_to_prefix
to handle negated terms correctly.
(imported from commit 673c0d3a5a77784e95772c14e12534ad2daecda2)
Extract 'verb' in Filter.operator_to_prefix() and
Filter.describe(). This doesn't change any functionality;
it is designed to make a subsequent change have a less
noisy diff.
(imported from commit e9c4b6edc498d88ec1783ccdba079d980def9438)
The methods describe() and operator_to_prefix() are
now in filter.js. They seem like they eventually
may be useful outside the scope of search suggestions,
and they seem to go hand in hand with other Filter
responsibilities.
(imported from commit 55f14c3ee848febb2e7c8c6a27afd690569b4e08)
Commit "ecf0eb85 Redesign styles for message pane" removed the
right_part class, updates the tests to not use it.
(imported from commit 277eb3748913895b13ab7bdca11e668033c9f9b3)
Also:
* Change fixed element positioning and fix bugs
* Move settings dropdown back to the right and add left padding to left sidebar
(imported from commit fcf903b59617687f94618a01ce7544b69f408130)
We now allow the list of recipients to be sent as a
comma-delimited string with optional JSON encoding.
(imported from commit e928b037bbd258348eb5b2ecca486d0bb77f593e)
Remove the options to narrow by topic/person from the menu,
because there are better ways to do this in the UI, and
remove the time travel option, because the "Link to this
conversation" achieves mostly the same effect.
(imported from commit b7e0cfe64c0760e5a7bf7a8c9c05ed1a5b747300)
Adding one row to the Navigation table made the
Keyboard shortcuts dialog look ridiculous, and it
was caused by a design based around float-left-ing
that broke down when the size of the "Navigation" box
was bigger than the "Composing messages" box.
Now we use a div to enclose the top two boxes in
the modal.
(imported from commit 90288ec288d4cf3d50ed7f5bcb48c8bb3d033f19)
All usages of json_to_dict were replaced with the check_dict
validator. The check_dict validations can eventually be
extended to validate the keys and values of incoming data,
but now we just use check_dict([]) in all the places where
we had json_to_dict, which means we aren't checking for any
specific keys; we are just making sure it's a dictionary.
(imported from commit fc5add9a7ef149dfac2a9a6d9a153799c4c0c24d)
We now will match an alert word even if it is used at the boundry of
bolding, backtick escaping, or caret quoting.
Closes trac #2186.
(imported from commit 984bc63eb621772c95a01ca5c5bfeb190767f71f)
This reverts commit 462a3eb5e6b83f9d8091b83e3f8dc458236938ed.
We're reverting this to see if it is the cause of our recent CSS
performance issues.
(imported from commit 6a0b041cfcb6770bbfda0d354444bad2d64459ab)
For the Filter helper functions above, we generally want to
ignore negated search terms, since their existence should
really only impact filter predicates and nothing else on the
JS side. The exception is search, where even the existence
of a negated search needs to be noted to know that we can't
apply a filter locally.
(imported from commit 8bbb410a85fefed549d359e4c779a134ad830c11)
For negated search terms, we weren't explicitly setting
"negated" to false when callers left it undefined, which was
mostly fine, since undefined is falsey, but it is better to
define it explicitly for debugging/testing purposes.
(imported from commit 68a2790b510d17caed8ca11c38188545d1dcc347)
Behind a feature flag you can now do searches like this:
-pm-with:othello@example.com is:private
The "-" in front of "pm-with" tells us to exclude messages
with Othello from our search. We support "-" in front of
all operators, although the behavior for "-search:" and
and "-near:" doesn't really change in this commit.
Note that the filtering out of "negated" predicates only
happens on the client side in this commit. On the server
side we ignore negated predicates and send back a superset
of the results.
(imported from commit 6cdeaf32f2d493fbbb838630f0da3da880b1ca18)
To deploy this, the zulip_internal::base and zulip_internal::munin classes must
be added to nagios.zulip.net.
(imported from commit 50d6a4ed19fcc9c62c7104977d69043bf5b9bbf9)
IE sometimes returns the pathname without a leading slash. Also
location.origin is not supported and must be build manually.
(imported from commit fb64478aeaac0f17d31021b7c370ff56781b48d1)
This commit doesn't change any functionality, and it is
designed to make diffs for upcoming changes related to
negated conditions a bit easier to read. This diff
looks a bit noiser than it really is due to some
reindentation of continuation lines.
(imported from commit 64c1cba98faa4bad4eaad122dd3de119caa880c0)
This this removed one forced relayout of the page on unnarrow. This
saves about 100ms for me.
(imported from commit 0755f425abbe3d99b8a99765549a5bbf3c620b9a)
Previously, we saved the current_msg_list selected id and then
restored it as the home_msg_list selected id, which could result in
the home view loading to the wrong place.
This takes some already bad code and makes it even more in need of
refactoring, but it does fix a pressing bug. We should definitely
refactor both:
* the top of narrow.js
* the save/restore code in reload.js
after this, though.
(imported from commit bb2040219e4f545ba90bb04a696996cec2831484)
This makes it simpler to test between two VMs by allowing you to bind to
non localhost interfaces.
(imported from commit f70755533b52ff8c49fd916941d2210fb8c33b47)
Browsing this list can be a great way to find feature ideas to
implement that other Zulip users are excited about.
* [2016 roadmap milestone](http://zulip.readthedocs.io/en/latest/roadmap.html): The
projects that are [priorities for the Zulip project](https://zulip.readthedocs.io/en/latest/roadmap.html). These are great projects if you're looking to make an impact.
If you're excited about helping with an open issue, just post on the
conversation thread that you're working on it. You're encouraged to
ask questions on how to best implement or debug your changes -- the
Zulip maintainers are excited to answer questions to help you stay
unblocked and working efficiently.
We also welcome suggestions of features that you feel would be
valuable or changes that you feel would make Zulip a better open
source project, and are happy to support you in adding new features or
other user experience improvements to Zulip.
If you have a new feature you'd like to add, we recommend you start by
opening a GitHub issue about the feature idea explaining the problem
that you're hoping to solve and that you're excited to work on it. A
Zulip maintainer will usually reply within a day with feedback on the
idea, notes on any important issues or concerns, and and often tips on
how to implement or test it. Please feel free to ping the thread if
you don't hear a response from the maintainers -- we try to be very
responsive so this usually means we missed your message.
For significant changes to the visual design, user experience, data
model, or architecture, we highly recommend posting a mockup,
screenshot, or description of what you have in mind to zulip-devel@ to
get broad feedback before you spend too much time on implementation
details.
Finally, before implementing a larger feature, we highly recommend
looking at the new feature tutorial and coding style guidelines on
ReadTheDocs.
Feedback on how to make this development process more efficient, fun,
and friendly to new contributors is very welcome! Just send an email
to the Zulip Developers list with your thoughts.
License
=======
Copyright 2011-2015 Dropbox, Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
The software includes some works released by third parties under other
free and open source licenses. Those works are redistributed under the
license terms under which the works were received. For more details,
see the ``THIRDPARTY`` file included with this distribution.
$(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/)
|``zerver/management/commands/`` | Management commands one might run at a production deployment site (e.g. scripts to change a value or deactivate a user properly) |
This page documents additional information that may be useful when developing new features for Zulip that require front-end changes. For a more general overview, see the new feature tutorial. The code style documentation also has relevant information about how Zulip's code is structured.
Primary build process
=====================
Most of the existing JS in Zulip is written in IIFE-wrapped modules,
one per file in the `static/js` directory. When running Zulip in
development mode, each file is loaded seperately. In production mode
(and when creating a release tarball using
`tools/build-release-tarball`), JavaScript files are concatenated and
minified.
If you add a new JavaScript file, it needs to be specified in the
`JS_SPECS` dictionary defined in `zproject/settings.py` to be included
in the concatenated file.
Webpack/CommonJS modules
========================
New JS written for Zulip can be written as CommonJS modules (bundled
using `webpack <https://webpack.github.io/>`_, though this will taken
care of automatically whenever ``run-dev.py`` is running). (CommonJS
is the same module format that Node uses, so see `the Node
documentation <https://nodejs.org/docs/latest/api/modules.html>` for
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to Zulip documentation!
===============================
Contents:
..toctree::
:maxdepth:2
integration-guide
new-feature-tutorial
directory-structure
code-style
testing
markdown
queuing
schema-migrations
front-end-build-process
mypy
translating
roadmap
Indices and tables
==================
*:ref:`genindex`
*:ref:`modindex`
*:ref:`search`
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
