Repository files navigation

Artificial Text Detection

Python framework for artificial text detection:NLP approaches to compare natural text against generated by neural networks.

Project description is put into:

• Framework Description Markdown
• Data Description Markdown
• Models Description Markdown

We usepoetry as an enhanced dependency resolver.

make poetry-downloadpoetry install --no-dev

To create datasets for the further classification, it is necessary to collect them.There are 2 available ways for it:

• ViaData Version Control.Get in touch with@msaidov in order to have the access to the private Google Drive;
• Via datasets generation. One dataset with a size of 20,000 samples was process with MT model on V100 GPU for 30 mins;

poetry add"dvc[gdrive]"

Then, rundvc pull. It will download preprocessed translation datasetsfrom the Google Drive.

To generate translations before artificial text detection pipeline,install thedetection module from the cloned repo or PyPi (TODO):

pip install -e.

Then, run generate script:

python detection/data/generate.py --dataset_name='tatoeba' --size=20000 --device='cuda:0'

To run the artificial text detection classifier, execute the pipeline:

python detection/old.py