Repository navigation
Attribute values aren't normalized properly #371
Description
Section 3.3.3 of the XML spec (the attribute value normalization section) defines that
For each character, entity reference, or character reference in the unnormalized attribute value, beginning with the first and continuing to the last, do the following:
For a character reference, append the referenced character to the normalized value.
For an entity reference, recursively apply step 3 of this algorithm to the replacement text of the entity.
For a white space character (#x20, #xD, #xA, #x9), append a space character (#x20) to the normalized value.
For another character, append the character to the normalized value.
If the attribute type is not CDATA, then the XML processor MUST further process the normalized attribute value by discarding any leading and trailing space (#x20) characters, and by replacing sequences of space (#x20) characters by a single space (#x20) character.
Note that if the unnormalized attribute value contains a character reference to a white space character other than space (#x20), the normalized value contains the referenced character itself (#xD, #xA or #x9). This contrasts with the case where the unnormalized value contains a white space character (not a reference), which is replaced with a space character (#x20) in the normalized value and also contrasts with the case where the unnormalized value contains an entity reference whose replacement text contains a white space character; being recursively processed, the white space character is replaced with a space character (#x20) in the normalized value.
All attributes for which no declaration has been read SHOULD be treated by a non-validating processor as if declared CDATA.
It is an error if an attribute value contains a reference to an entity for which no declaration has been read.
TL;DR: Bullet point 3 states that 0xD, 0xA, and 0x9 (\r, \n and \t) within an attribute value ought to be treated as whitespace and replaced with a whitespace character (0x20), unless the character was explicitly referenced.
https://www.w3.org/TR/xml/#AVNormalize
Here's a half-written test which demonstrates the failure:
#[test]
fn test_attribute_value_normalization() {
let mut r = Reader::from_str("<a attr=\"tab character\tor newline\nor return\rshould be replaced\">");
r.trim_text(true);
let mut buf = Vec::new();
match r.read_event(&mut buf) {
Ok(Start(e)) => {
let mut attrs = e.attributes();
match attrs.next() {
Some(Ok(attr)) => assert_eq!(attr, ("attr".as_bytes(), "tab character or newline or return should be replaced".as_bytes()).into()),
x => panic!("expected attribute 'attr', got {:?}", x),
}
}
x => panic!("expected <a attr='>'>, got {:?}", x),
}
}---- test_attribute_value_normalization stdout ----
thread 'test_attribute_value_normalization' panicked at 'assertion failed: `(left == right)`
left: `Attribute { key: "attr", value: Borrowed("tab character0x9or newline0xAor return0xDshould be replaced") }`,
right: `Attribute { key: "attr", value: Borrowed("tab character or newline or return should be replaced") }`', tests/unit_tests.rs:850:35
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Activity
Properly normalize attribute values
Properly normalize attribute values
Properly normalize attribute values
Properly normalize attribute values
Properly normalize attribute values
Properly normalize attribute values
Properly normalize attribute values
Properly normalize attribute values
Properly normalize attribute values
Properly normalize attribute values
dralley commentedon Jun 20, 2022
HTML doesn't seem to do any attribute normalization, so we ought to take that into account.
Properly normalize attribute values
Properly normalize attribute values
Properly normalize attribute values
Properly normalize attribute values
7 remaining items
Properly normalize attribute values
Add functions for attribute value normalization
Add functions for attribute value normalization
Add functions for attribute value normalization
Add functions for attribute value normalization
Add functions for attribute value normalization
Add functions for attribute value normalization
Add functions for attribute value normalization
Add functions for attribute value normalization
Add functions for attribute value normalization
Add functions for attribute value normalization
Add functions for attribute value normalization
Add functions for attribute value normalization
Add functions for attribute value normalization