beimingwu
/
learnware
Not watched
1
0
Fork 0
Code
Releases 1 Wiki evaluate Activity
Issues 0 Pull Requests 0 Datasets Model Cloudbrain HPC
You can not select more than 25 topics Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.
Tree: a088f6b099
Branches Tags
${ item.name }
Create branch ${ searchTerm }
from 'a088f6b099'
${ noResults }
learnware/examples/dataset_text_workflow/get_data.py

18 lines
637 B
Raw Blame History 

  1. import os

  2. import json

  3. import numpy as np

  4. from sklearn.datasets import fetch_20newsgroups

  5. import pandas as pd

  6. 

  7. def get_data(data_root):

  8.     dataset_train = fetch_20newsgroups(data_home=data_root, subset='train')

  9.     target_names = dataset_train["target_names"]

  10. 

  11.     X_train = np.array(dataset_train["data"])

  12.     y_train = pd.Categorical.from_codes(dataset_train["target"], categories=target_names)

  13. 

  14.     X_test, y_test = fetch_20newsgroups(data_home=data_root, subset='test', return_X_y=True)

  15.     X_test = np.array(X_test)

  16.     y_test = pd.Categorical.from_codes(y_test, categories=target_names)

  17. 

  18.     return X_train, y_train, X_test, y_test