No contracts You may keep provided that you want or terminate Anytime. You obtain to help make that choice, provided that you are not financing a tool. txt file is then parsed and will instruct the robot as to which webpages are not for being crawled. Like a internet search https://www.youtube.com/watch?v=MxDLv74CqK0