58c20698d6
Because some tokens can contain newlines and not only comment tokens, I removed the comments specific code to handle newlines in a more generic way.