documentation_crawler: Exclude communities page.

Since /communities/#all was failing the spider test due to `#all`
not being present as an `id` on the page, we exclude it from
this check saying that it is not really a documentation page.
This commit is contained in:
Aman Agrawal
2022-10-20 20:08:27 +00:00
committed by Tim Abbott
parent 6266954229
commit 587f52e351

View File

@@ -146,6 +146,11 @@ class BaseDocumentationSpider(scrapy.Spider):
): ):
return return
# This page has some invisible to the user anchor links like #all
# that are currently invisible, and thus would otherwise fail this test.
if url.startswith("http://localhost:9981/communities"):
return
callback: Callable[[Response], Optional[Iterator[Request]]] = self.parse callback: Callable[[Response], Optional[Iterator[Request]]] = self.parse
dont_filter = False dont_filter = False
method = "GET" method = "GET"