Due to a missing `not`, this incorrectly skipped checking fragments
for everything *other* than the /help documentation!
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Commits d333ddb961 and
a766c092fc (#35621) broke this by
changing the base class from UnusedImagesLinterSpider to
BaseDocumentationSpider.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
We add a step to build help center and then test the broken links as we
used to before removing the test temporarily.
This commit focuses on just adding back the broken link checks for the
help center. We skip the fragment check since that is in-built in
starlight and starlight tests account for that already. For the image
check we can add it back in a followup issue.
This commit removes the current help center markdown files and any logic
that was used to host those files at help/.
We also remove a bunch of tests, we should the equivalent of those tests
for the new help center. Issues to track: #35649, #35647. These issues
track adding back tests for redirects and broken links.
We had a symlink from templates/zerver/integrations/email.md pointing to
help/using-zulip-via-email.md. We can no longer have that symlink since
the latter has been converted to an MDX file. We have deleted the
symlink and put a markdown file in it's place. Both the files have
comments to edit the other in case of changes.
This commit also makes changes in astro config, astro component paths
and other places to move the starlight help center docs base path from
/starlight_help to /help.
The change to rename /starlight_help/ to /help/ in MDX files is done in
the next commit. If we squash these commits, this line should be
removed.
`./tools/build-help-center` no longer does the conversion step.
We also remove some dead code related to the old help center in
documentation.py.
In 2f547ea, the custom `has_error` logic in BaseDocumentationSpider
was removed in favor of checking whether any errors were logged.
One reference to that method was missed in UnusedImagesLinterSpider
and went unremarked as no images were added to the documentation
that were not used.
Replaces that reference with an error log.
Fixes a warning caused by using the deprecated `2.6` value
of the `REQUEST_FINGERPRINTER_IMPLEMENTATION` setting.
Upgrades `settings.py` to what would have been generated
by starting the documentation_crawler project using the
`startproject` command of scrapy v2.7.
Since /communities/#all was failing the spider test due to `#all`
not being present as an `id` on the page, we exclude it from
this check saying that it is not really a documentation page.
I have suppressed errors for github.com by adding an function to
exclude domains as well as urls; this is necessary because GitHub has
marked this tool's User-Agent as a blocker crawler.
I have also suppressed reoccurring url errors that do definitely exist.
Fixes#17928.
