Table of Contents

Dataset Description

Dataset Summary

Supported Tasks and Leaderboards

Languages

Dataset Structure

Data Instances

fromdatasetsimportload_datasetdataset=load_dataset("shunk031/livedoor-news-corpus",train_ratio=0.8,val_ratio=0.1,test_ratio=0.1,random_state=42,shuffle=True,)print(dataset)# DatasetDict({#     train: Dataset({#         features: ['url', 'date', 'title', 'content', 'category'],#         num_rows: 5894#     })#     validation: Dataset({#         features: ['url', 'date', 'title', 'content', 'category'],#         num_rows: 737#     })#     test: Dataset({#         features: ['url', 'date', 'title', 'content', 'category'],#         num_rows: 736#     })# })

Data Fields

Data Splits

Dataset Creation

Curation Rationale

Source Data

Initial Data Collection and Normalization

Who are the source language producers?

Annotations

Annotation process

Who are the annotators?

Personal and Sensitive Information

Considerations for Using the Data

Social Impact of Dataset

Discussion of Biases

Other Known Limitations

Additional Information

Dataset Curators

Licensing Information

Citation Information

Contributions