Many servers will return errors (e.g. 400/403) to requests that do not
set a User-Agent header. This results in issues in both the link_checker
and load_data components. With the link_checker these are false positive
dead links. In load_data, remote data fails to be fetched. To mitigate
this issue, this sets a default User-Agent of
$CARGO_PKG_NAME/$CARGO_PKG_VERSION
Note that the root cause of this regression from zola v0.9.0 is that
reqwest 0.10 changed their default behavior and no longer sets a
User-Agent by default:
https://github.com/seanmonstar/reqwest/pull/751Fixes#950.
For the site integration tests, we have a file of common code which is
used by multiple files in `tests/`. However, not all functions in
this file are used by all files in `tests/`.
As Cargo compiles each `tests/*.rs` file as a separate crate, this
means that some of these crates end up with unused code. Rust notices
this and prints a warning.
Let's tell Rust that we don't care about dead code in this file so
that the warning is not printed.
* Detect empty links on markdown rendering and issue an error
* Add a test for empty links stopping rendering with an error
* Assert error message is the expected one
When testing for empty links detection compare the error message
to make sure it's the correct error that stopped the process
and not some unrelated issue.
The issue with the check_site test hanging and timing out seems to
be related to a similar reqwest issue, which was ultimately due to
an upstream bug in tokio and may be fixed in tokio 0.2.7 onward.
* Restore #![feature(test)] and extern crate test; statements, which
were mistakenly removed as part of the Rust 2018 edition migration.
* Fix rendering benchmark's usage of RenderContext. 6 parameters were
provided when 5 were expected.
* Treat 304 (Not Modified) requests as valid.
* Add tests for 301-to-200 links, 301-to-404 links, and 500 links.
This helps to test redirections and the previously-added
response.status() checking for non-success status codes in check_url().
* Make names for HTTP mock paths unique, to avoid weird behavior. It
seems like mocks with the same path can potentially bleed between
tests, so you may end up with an unexpected response which causes the
test to sometimes pass and sometimes fail.
* Fix Clippy warnings about String::from(format!()).
Certain tests involving HTTP requests were sometimes hanging
indefinitely, so this uses Mockito for HTTP mocking. This seemingly
resolves the issue and makes these tests more reliable.
The existing can_fail_404_links test has been renamed to
can_fail_unresolved_links, to represent what actually occurs in the
test. The can_fail_404_links test now deals with a proper 404
response.
Just to be clear, the check_site test in the site component will
still create outgoing HTTP requests (due to the URLs used in the
test_site), so this commit only uses HTTP mocking where possible.
The can_fail_404_links() test doesn't encounter a 404 response in
actuality, since the google.comys domain doesn't resolve. When the
test is updated such that the response's status code is a 404, the
test fails because the check_url() function doesn't handle
non-success responses how the test's assertions expect. This commit
updates check_url() to handle non-success responses, treating them
much like errors.
* maybe_slugify() only does simple sanitation if config.slugify is false
* slugify is disabled by default, turn on for backwards-compatibility
* First docs changes for optional slugification
* Remove # from slugs but not &
* Add/fix tests for utf8 slugs
* Fix test sites for i18n slugs
* fix templates tests for i18n slugs
* Rename slugify setting to slugify_paths
* Default slugify_paths
* Update documentation for slugify_paths
* quasi_slugify removes ?, /, # and newlines
* Remove forbidden NTFS chars in quasi_slugify()
* Slugification forbidden chars can be configured
* Remove trailing dot/space in quasi_slugify
* Fix NTFS path sanitation
* Revert configurable slugification charset
* Remove \r for windows newlines and \t tabulations in quasi_slugify()
* Update docs for output paths
* Replace slugify with slugify_paths
* Fix test
* Default to not slugifying
* Move slugs utils to utils crate
* Use slugify_paths for anchors as well
* Add path to `TranslatedContent`
This makes it possible to retrieve the translated page through the `get_page` function.
* Use TranslatedContent::path field in test_site_i18n
Use it with the `get_page` function to get a reference to the page object.
* Compute canonical path before adjusting parent path
* Don't use adjusted `parent` to recalculate `canonical` in `find_language`
* Add regression tests
- Test for correct canonical field when calling `new_page`
- Test for correct canonical field after calling `find_language`
* feat(pagination): Add `total_pages` in paginator object
* feat(pagination): Added doc for `total_pages`
* feat(pagination): Added test for `total_pages`
"[…] `&` normally indicates the start of a character entity reference or
numeric character reference; writing it as `&` […] allows `&` to be
included in the content of an element or in the value of an attribute."
From: https://en.wikipedia.org/wiki/HTML#Character_and_entity_references
Links that start with a scheme (e.g., `tel:18008675309`) inadvertently
had a URL prefix prepended. Previously, only `mailto:` was handled, but
given the sheer number of [registered URI schemes][uri-schemes], a loose
pattern matcher is used to detect schemes instead.
External links, as identified by the renderer, are now limited to `http`
and `https` schemes.
Fixes#747 and fixes#816.
[uri-schemes]: https://www.iana.org/assignments/uri-schemes/uri-schemes.xhtml
* Add anchor existant checking to link_checker component
* Oops, forgot some changes
* Drop scraper dependency and rework tests
* Handle name attributes
The access to translations is not straightforward and requires checks if
language and key exists. It is better to forbit direct access to
attribute and provide method - `get_translation()` - that will handle
all details of key translations.
Remove unit tests that use direct access and test only public method.
