DataSetIteratorを使用してTransformProcessデータを変換する

Question

数値属性と名義属性の両方を持つCSVデータセットがあります。私は、公称属性のすべての可能な値をリストするデータセットに対してスキーマを定義しました。その後、TransformProcessを作成して、CategoricalToOneHotTransformを使用して公称値を数値に変換しました。どのように私はこれを使用することができますTransformProcessRecordReaderDataSetIterator私のニューラルネットワークを準備するには？

 Schema schema = new Schema.Builder() 
     .addColumnInteger("age") 
     .addColumnCategorical("workclass", "Private", "Self-emp-not-inc", "Self-emp-inc", "Federal-gov", "Local-gov", "State-gov", "Without-pay", "Never-worked") 
     .addColumnInteger("fnlwgt") 
     .addColumnCategorical("education", "Bachelors", "Some-college", "11th", "HS-grad", "Prof-school", "Assoc-acdm", "Assoc-voc", "9th", "7th-8th", "12th", "Masters", "1st-4th", "10th", "Doctorate", "5th-6th", "Preschool") 
     .addColumnInteger("education-num") 
     .addColumnCategorical("marital-status", "Married-civ-spouse", "Divorced", "Never-married", "Separated", "Widowed", "Married-spouse-absent", "Married-AF-spouse") 
     .addColumnCategorical("occupation", "Tech-support", "Craft-repair", "Other-service", "Sales", "Exec-managerial", "Prof-specialty", "Handlers-cleaners", "Machine-op-inspct", "Adm-clerical", "Farming-fishing", "Transport-moving", "Priv-house-serv", "Protective-serv", "Armed-Forces") 
     .addColumnCategorical("relationship", "Wife", "Own-child", "Husband", "Not-in-family", "Other-relative", "Unmarried") 
     .addColumnCategorical("race", "White", "Asian-Pac-Islander", "Amer-Indian-Eskimo", "Other", "Black") 
     .addColumnCategorical("sex", "Female", "Male") 
     .addColumnInteger("capital-gain") 
     .addColumnInteger("capital-loss") 
     .addColumnInteger("hours-per-week") 
     .addColumnCategorical("native-country", "United-States", "Cambodia", "England", "Puerto-Rico", "Canada", "Germany", "Outlying-US(Guam-USVI-etc)", "India", "Japan", "Greece", "South", "China", "Cuba", "Iran", "Honduras", "Philippines", "Italy", "Poland", "Jamaica", "Vietnam", "Mexico", "Portugal", "Ireland", "France", "Dominican-Republic", "Laos", "Ecuador", "Taiwan", "Haiti", "Columbia", "Hungary", "Guatemala", "Nicaragua", "Scotland", "Thailand", "Yugoslavia", "El-Salvador", "Trinadad&Tobago", "Peru", "Hong", "Holand-Netherlands") 
     .addColumnCategorical("class", ">50K", "<=50K") 
     .build(); 

    TransformProcess tp = new TransformProcess.Builder(schema) 
     .transform(new CategoricalToOneHotTransform("workclass")) 
     .transform(new CategoricalToOneHotTransform("education")) 
     .transform(new CategoricalToOneHotTransform("marital-status")) 
     .transform(new CategoricalToOneHotTransform("occupation")) 
     .transform(new CategoricalToOneHotTransform("relationship")) 
     .transform(new CategoricalToOneHotTransform("race")) 
     .transform(new CategoricalToOneHotTransform("sex")) 
     .transform(new CategoricalToOneHotTransform("native-country")) 
     .transform(new CategoricalToIntegerTransform("class")) 
     .build(); 

    Schema outputSchema = tp.getFinalSchema(); 

    int numLinesToSkip = 0; 
    String delimiter = ","; 
    CSVRecordReader recordReader = new CSVRecordReader(numLinesToSkip, delimiter); 
    recordReader.initialize(new FileSplit(Paths.get("..\\adult.data").toFile())); 


    int labelIndex = outputSchema.getColumnNames().size() - 1; 
    int numClasses = 2; 
    int batchSize = 2000; 

    RecordReaderDataSetIterator iterator = new RecordReaderDataSetIterator(recordReader, batchSize, labelIndex, numClasses); 

    DataSet allData = iterator.next(); 
    allData.shuffle(); 
    SplitTestAndTrain testAndTrain = allData.splitTestAndTrain(0.65); 

Answer 1

は見てみましょう：RecordReaderDataSetItertorは記録的リーダーで取り込んでベクトル化処理を扱うhttps://github.com/deeplearning4j/DataVec/blob/master/datavec-api/src/main/java/org/datavec/api/records/reader/impl/transform/TransformProcessRecordReader.java

。これは、レコードリーダをラップし、変換されたレコードを出力します。変換されたレコードは、レコードリーダーデータセレクタに送られます。