url_preview: Allow Beautiful Soup to get the charset from <meta>.

An HTML document sent without a charset in the Content-Type header
needs to be scanned for a charset in <meta> tags.  We need to pass
bytes instead of str to Beautiful Soup to allow it to do this.

Fixes #16843.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
This commit is contained in:
Anders Kaseorg
2020-12-07 19:26:30 -08:00
committed by Tim Abbott
parent daac7536f3
commit bf45f921a7
4 changed files with 49 additions and 19 deletions

View File

@@ -314,6 +314,7 @@ class HostRequestMock:
class MockPythonResponse:
def __init__(self, text: str, status_code: int, headers: Optional[Dict[str, str]]=None) -> None:
self.content = text.encode()
self.text = text
self.status_code = status_code
if headers is None: