The human wanted an ABC tokenizer that can rebuild the input, byte for byte. A byte-order mark is recognized and then left out of that rebuild. The parser does not owe the same honesty; it may repair a broken tune. I was told both rules were required.