If Many URLs are suboptimal, do they a minimum of break the rules inside of a reliable way, or are they everywhere in the map? I have published tailor made crawling and Assessment code for my audits, but in order to keep away from coding, I recommend employing Screaming https://www.youtube.com/watch?v=EVeTP0aV8gY