{% extends "zerver/portico.html" %}

{% block title %}
<title>Tools and data sets | Zulip Dev</title>
{% endblock %}

{# Login page. #}
{% block portico_content %}

<div class="app flex full-page">
    <div id="devtools-page" class="markdown">
        <h1>Useful development URLs</h1>
        <p>
            Below is a list of useful tools and data sets available only in the Zulip
            development environment that are often useful when contributing to Zulip.
            Most of these require you to run a command to build/generate the relevant
            content. This table specifies which command to use to update the data served
            by each page (since several of these, like test coverage, require a special
            command to be run to generate the data). Make sure your development server is still running
            when you visit these!
        </p>
        <table class="table table-striped table-rounded table-bordered">
            <thead>
                <tr>
                    <th>URL</th>
                    <th>Command</th>
                    <th>Description</th>
                </tr>
            </thead>
            <tbody>
                <tr>
                    <td><a href="/coverage/index.html">/coverage/index.html</a></td>
                    <td><code>./tools/test-backend --coverage</code></td>
                    <td>Backend (Django) test coverage report</td>
                </tr>
                <tr>
                    <td><a href="/node-coverage/index.html">/node-coverage/index.html</a></td>
                    <td><code>./tools/test-js-with-node --coverage</code></td>
                    <td>Frontend (node) test coverage report</td>
                </tr>
                <tr>
                    <td><a href="/docs/index.html">/docs/index.html</a></td>
                    <td><code>./tools/build-docs</code></td>
                    <td>Developer documentation (ReadTheDocs) built locally</td>
                </tr>
                <tr>
                    <td><a href="/emails">/emails</a></td>
                    <td>View outgoing and example emails.</td>
                </tr>
                <tr>
                    <td><a href="/stats/realm/analytics/">/stats/realm/analytics/</a></td>
                    <td><code>./manage.py populate_analytics_db</code><br />
                        Run the command after changing analytics data population logic.
                    </td>
                    <td>View the /stats page with some pre-populated data</td>
                </tr>
                <tr>
                    <td><a href="/webpack/5xx.html">/webpack/5xx.html</a></td>
                    <td>None needed</td>
                    <td>Error 5xx page served by nginx (used when Django is totally broken)</td>
                </tr>
                <tr>
                    <td><a href="/errors/404">/errors/404</a></td>
                    <td>None needed</td>
                    <td>Error 404 page served by Django</td>
                </tr>
                <tr>
                    <td><a href="/errors/5xx">/errors/5xx</a></td>
                    <td>None needed</td>
                    <td>Error 5xx page served by Django</td>
                </tr>
                <tr>
                    <td><a href="/accounts/do_confirm/invalid">/accounts/do_confirm/invalid</a></td>
                    <td>None needed</td>
                    <td>Invalid confirmation link page</td>
                </tr>
                <tr>
                    <td><a href="/devtools/integrations">/devtools/integrations</a></td>
                    <td>None needed</td>
                    <td>Test incoming webhook integrations</td>
                </tr>
            </tbody>
        </table>
        <h2>Useful management commands</h2>
        <p>Development-specific <a href="https://zulip.readthedocs.io/en/latest/production/management-commands.html">management commands</a> live in <code>zilencer/management/commands</code>.  Highlights include:
            <ul>
                <li><code>./manage.py populate_db</code>: Rebuilds database. Has options to, for example, create 3K users for testing.</li>
                <li><code>./manage.py mark_all_messages_unread</code>: Useful for testing reading messages.</li>
                <li><code>./manage.py create_realm</code>: Add a new realm. Useful for testing onboarding.</li>
                <li><code>./manage.py create_user</code>: Add a new user. Useful for testing onboarding.</li>
                <li><code>./manage.py add_mock_conversation</code>: Add test messages, streams, images, emoji, etc.
                    into the dev environment. First edit zilencer/management/commands/add_mock_conversation.py
                    to add the data you're testing.
                </li>
            </ul>
        </p>
        <p>We also have
            <a href="https://zulip.readthedocs.io/en/latest/development/authentication.html">documentation on testing LDAP, Google &amp; GitHub authentication</a> in the development environment.
        </p>
        <h2>Connecting to the local PostgreSQL database</h2>
        <ul>
            <li>
                <code>./manage.py dbshell</code>: Connect to
                PostgreSQL database via your terminal.
            </li>
            <li>
                <code>provision</code> creates a <code>~/.pgpass</code> file,
                so <code>psql -U zulip -h localhost</code> works too.
            </li>
            <li>
                <p>
                    To connect using a graphical PostgreSQL client
                    like <a href="https://www.pgadmin.org/">pgAdmin</a>,
                    use the following credentials:
                </p>
                <ul>
                    <li>Host: 127.0.0.1 (don't use localhost)</li>
                    <li>Maintenance database: zulip</li>
                    <li>Username: zulip</li>
                    <li>password: stored as <code>local_database_password</code> in <code>zulip/zproject/dev-secrets.conf</code></li>
                </ul>
            </li>
        </ul>
        <h2>Development instructions for Help Center (Beta)</h2>
        <p>
            These commands are to run the project for an ongoing migration for our help center docs to use
            <a href="https://github.com/withastro/starlight">@astrojs/starlight</a>. You can track the
            progress for the project at <a href="https://github.com/zulip/zulip/issues/30450">#30450</a>.
        </p>
        <ul>
            <li>
                <code>./tools/build-help-center</code> will convert the existing help center MD files to MDX.
                It will also create a production build for the converted MDX files. You can run
                <code>./tools/convert-help-center-docs-to-mdx</code> to convert the files separately.
                You can run <code>pnpm build</code> within the <code>help-beta</code> directory
                to run the build step separately.
            </li>
            <li>
                <code>./tools/run-dev --help-center</code> will host the generated build on
                <code>/help-beta</code> url. You will need to have run the build steps before this step.
                The docs search will work with this flag.
            </li>
            <li>
                <code>./tools/run-dev --help-center-dev-server</code> will run a dev server at
                <code>/help-beta</code> that supports hot reload. But, with this flag, the search will not work.
                 This mode is ideal when you&#39;re editing a document and want to visualize the changes quickly.
                You will need the converted MDX files to be already generated before you run this step.
            </li>
        </ul>
    </div>
</div>

{% endblock %}