标签：文件路径 import 标准 model imp OLE use bfc standard
Java 类名：com.alibaba.alink.operator.batch.dataproc.StandardScalerTrainBatchOp
Python 类名：StandardScalerTrainBatchOp
标准化是对数据进行按正态化处理的组件
训练过程计算数据的均值和标准差，在预测组件中使用模型结果
|
名称 |
中文名称 |
描述 |
类型 |
是否必须？ |
默认值 |
|
selectedCols |
选择的列名 |
计算列对应的列名列表 |
String[] |
? |
|
|
withMean |
是否使用均值 |
是否使用均值，默认使用 |
Boolean |
true |
|
|
withStd |
是否使用标准差 |
是否使用标准差，默认使用 |
Boolean |
true |
from pyalink.alink import * import pandas as pd useLocalEnv(1) df = pd.DataFrame([ ["a", 10.0, 100], ["b", -2.5, 9], ["c", 100.2, 1], ["d", -99.9, 100], ["a", 1.4, 1], ["b", -2.2, 9], ["c", 100.9, 1] ]) colnames = ["col1", "col2", "col3"] selectedColNames = ["col2", "col3"] inOp = BatchOperator.fromDataframe(df, schemaStr=‘col1 string, col2 double, col3 long‘) # train trainOp = StandardScalerTrainBatchOp() .setSelectedCols(selectedColNames) trainOp.linkFrom(inOp) # batch predict predictOp = StandardScalerPredictBatchOp() predictOp.linkFrom(trainOp, inOp).print()
import org.apache.flink.types.Row; import com.alibaba.alink.operator.batch.BatchOperator; import com.alibaba.alink.operator.batch.dataproc.StandardScalerPredictBatchOp; import com.alibaba.alink.operator.batch.dataproc.StandardScalerTrainBatchOp; import com.alibaba.alink.operator.batch.source.MemSourceBatchOp; import org.junit.Test; import java.util.Arrays; import java.util.List; public class StandardScalerTrainBatchOpTest { @Test public void testStandardScalerTrainBatchOp() throws Exception { List <Row> df = Arrays.asList( Row.of("a", 10.0, 100), Row.of("b", -2.5, 9), Row.of("c", 100.2, 1), Row.of("d", -99.9, 100), Row.of("a", 1.4, 1), Row.of("b", -2.2, 9), Row.of("c", 100.9, 1) ); String[] selectedColNames = new String[] {"col2", "col3"}; BatchOperator <?> inOp = new MemSourceBatchOp(df, "col1 string, col2 double, col3 int"); BatchOperator <?> trainOp = new StandardScalerTrainBatchOp() .setSelectedCols(selectedColNames); trainOp.linkFrom(inOp); BatchOperator <?> predictOp = new StandardScalerPredictBatchOp(); predictOp.linkFrom(trainOp, inOp).print(); } }
|
col1 |
col2 |
col3 |
|
a |
-0.0784 |
1.4596 |
|
b |
-0.2592 |
-0.4814 |
|
c |
1.2270 |
-0.6521 |
|
d |
-1.6687 |
1.4596 |
|
a |
-0.2028 |
-0.6521 |
|
b |
-0.2549 |
-0.4814 |
|
c |
1.2371 |
-0.6521 |
Java 类名：com.alibaba.alink.operator.batch.dataproc.StandardScalerPredictBatchOp
Python 类名：StandardScalerPredictBatchOp
标准化是对数据进行按正态化处理的组件
使用标准化训练组件训练的模型，对数据做标准化处理
|
名称 |
中文名称 |
描述 |
类型 |
是否必须？ |
默认值 |
|
outputCols |
输出结果列列名数组 |
输出结果列列名数组，可选，默认null |
String[] |
null |
|
|
numThreads |
组件多线程线程个数 |
组件多线程线程个数 |
Integer |
1 |
|
|
modelStreamFilePath |
模型流的文件路径 |
模型流的文件路径 |
String |
null |
|
|
modelStreamScanInterval |
扫描模型路径的时间间隔 |
描模型路径的时间间隔，单位秒 |
Integer |
10 |
|
|
modelStreamStartTime |
模型流的起始时间 |
模型流的起始时间。默认从当前时刻开始读。使用yyyy-mm-dd hh:mm:ss.fffffffff格式，详见Timestamp.valueOf(String s) |
String |
null |
from pyalink.alink import * import pandas as pd useLocalEnv(1) df = pd.DataFrame([ ["a", 10.0, 100], ["b", -2.5, 9], ["c", 100.2, 1], ["d", -99.9, 100], ["a", 1.4, 1], ["b", -2.2, 9], ["c", 100.9, 1] ]) colnames = ["col1", "col2", "col3"] selectedColNames = ["col2", "col3"] inOp = BatchOperator.fromDataframe(df, schemaStr=‘col1 string, col2 double, col3 long‘) # train trainOp = StandardScalerTrainBatchOp() .setSelectedCols(selectedColNames) trainOp.linkFrom(inOp) # batch predict predictOp = StandardScalerPredictBatchOp() predictOp.linkFrom(trainOp, inOp).print()
import org.apache.flink.types.Row; import com.alibaba.alink.operator.batch.BatchOperator; import com.alibaba.alink.operator.batch.dataproc.StandardScalerPredictBatchOp; import com.alibaba.alink.operator.batch.dataproc.StandardScalerTrainBatchOp; import com.alibaba.alink.operator.batch.source.MemSourceBatchOp; import org.junit.Test; import java.util.Arrays; import java.util.List; public class StandardScalerPredictBatchOpTest { @Test public void testStandardScalerPredictBatchOp() throws Exception { List <Row> df = Arrays.asList( Row.of("a", 10.0, 100), Row.of("b", -2.5, 9), Row.of("c", 100.2, 1), Row.of("d", -99.9, 100), Row.of("a", 1.4, 1), Row.of("b", -2.2, 9), Row.of("c", 100.9, 1) ); String[] selectedColNames = new String[] {"col2", "col3"}; BatchOperator <?> inOp = new MemSourceBatchOp(df, "col1 string, col2 double, col3 int"); BatchOperator <?> trainOp = new StandardScalerTrainBatchOp() .setSelectedCols(selectedColNames); trainOp.linkFrom(inOp); BatchOperator <?> predictOp = new StandardScalerPredictBatchOp(); predictOp.linkFrom(trainOp, inOp).print(); } }
|
col1 |
col2 |
col3 |
|
a |
-0.0784 |
1.4596 |
|
b |
-0.2592 |
-0.4814 |
|
c |
1.2270 |
-0.6521 |
|
d |
-1.6687 |
1.4596 |
|
a |
-0.2028 |
-0.6521 |
|
b |
-0.2549 |
-0.4814 |
|
c |
1.2371 |
-0.6521 |
ALINK(二十)：数据处理（六）数值型数据处理（二）标准化 (StandardScalerPredictBatchOp/StandardScalerTrainBatchOp )
标签：文件路径 import 标准 model imp OLE use bfc standard
原文地址：https://www.cnblogs.com/qiu-hua/p/14897433.html
