Important Notice: Our web hosting provider recently started charging us for additional visits, which was unexpected. In response, we're seeking donations. Depending on the situation, we may explore different monetization options for our Community and Expert Contributors. It's crucial to provide more returns for their expertise and offer more Expert Validated Answers or AI Validated Answers. Learn more about our hosting issue here.

Why is there a –stop and a –nontoken option for count.pl?

April 26, 2017option

0

Posted

Why is there a –stop and a –nontoken option for count.pl?

2 Answers

0

Posted

The stop option allows you to specify a stop list that eliminates Ngrams if they are completely made up of stop words (AND mode) or if one of the words in the Ngram is a stop word (OR mode). The effect of the stop option is to remove Ngrams from the sample. The nontoken option allows you to eliminate words from the text prior to the formation of Ngrams. This processing occurs well before the stop option, which is carried out after Ngrams have been formed.

0

Posted

The stop option allows you to specify a stop list that eliminates Ngrams if they are completely made up of stop words (AND mode) or if one of the words in the Ngram is a stop word (OR mode). The effect of the stop option is to remove Ngrams from the sample.