Normalizer#
- classpyspark.ml.feature.Normalizer(*,p=2.0,inputCol=None,outputCol=None)[source]#
Normalize a vector to have unit norm using the given p-norm.
New in version 1.4.0.
Examples
>>>frompyspark.ml.linalgimportVectors>>>svec=Vectors.sparse(4,{1:4.0,3:3.0})>>>df=spark.createDataFrame([(Vectors.dense([3.0,-4.0]),svec)],["dense","sparse"])>>>normalizer=Normalizer(p=2.0)>>>normalizer.setInputCol("dense")Normalizer...>>>normalizer.setOutputCol("features")Normalizer...>>>normalizer.transform(df).head().featuresDenseVector([0.6, -0.8])>>>normalizer.setParams(inputCol="sparse",outputCol="freqs").transform(df).head().freqsSparseVector(4, {1: 0.8, 3: 0.6})>>>params={normalizer.p:1.0,normalizer.inputCol:"dense",normalizer.outputCol:"vector"}>>>normalizer.transform(df,params).head().vectorDenseVector([0.4286, -0.5714])>>>normalizerPath=temp_path+"/normalizer">>>normalizer.save(normalizerPath)>>>loadedNormalizer=Normalizer.load(normalizerPath)>>>loadedNormalizer.getP()==normalizer.getP()True>>>loadedNormalizer.transform(df).take(1)==normalizer.transform(df).take(1)True
Methods
clear(param)Clears a param from the param map if it has been explicitly set.
copy([extra])Creates a copy of this instance with the same uid and some extra params.
explainParam(param)Explains a single param and returns its name, doc, and optional default value and user-supplied value in a string.
Returns the documentation of all params with their optionally default values and user-supplied values.
extractParamMap([extra])Extracts the embedded default param values and user-supplied values, and then merges them with extra values from input into a flat param map, where the latter value is used if there exist conflicts, i.e., with ordering: default param values < user-supplied values < extra.
Gets the value of inputCol or its default value.
getOrDefault(param)Gets the value of a param in the user-supplied param map or its default value.
Gets the value of outputCol or its default value.
getP()Gets the value of p or its default value.
getParam(paramName)Gets a param by its name.
hasDefault(param)Checks whether a param has a default value.
hasParam(paramName)Tests whether this instance contains a param with a given (string) name.
isDefined(param)Checks whether a param is explicitly set by user or has a default value.
isSet(param)Checks whether a param is explicitly set by user.
load(path)Reads an ML instance from the input path, a shortcut ofread().load(path).
read()Returns an MLReader instance for this class.
save(path)Save this ML instance to the given path, a shortcut of 'write().save(path)'.
set(param, value)Sets a parameter in the embedded param map.
setInputCol(value)Sets the value of
inputCol.setOutputCol(value)Sets the value of
outputCol.setP(value)Sets the value of
p.setParams(self, \*[, p, inputCol, outputCol])Sets params for this Normalizer.
transform(dataset[, params])Transforms the input dataset with optional parameters.
write()Returns an MLWriter instance for this ML instance.
Attributes
Returns all params ordered by name.
Methods Documentation
- clear(param)#
Clears a param from the param map if it has been explicitly set.
- copy(extra=None)#
Creates a copy of this instance with the same uid and someextra params. This implementation first calls Params.copy andthen make a copy of the companion Java pipeline component withextra params. So both the Python wrapper and the Java pipelinecomponent get copied.
- Parameters
- extradict, optional
Extra parameters to copy to the new instance
- Returns
JavaParamsCopy of this instance
- explainParam(param)#
Explains a single param and returns its name, doc, and optionaldefault value and user-supplied value in a string.
- explainParams()#
Returns the documentation of all params with their optionallydefault values and user-supplied values.
- extractParamMap(extra=None)#
Extracts the embedded default param values and user-suppliedvalues, and then merges them with extra values from input intoa flat param map, where the latter value is used if there existconflicts, i.e., with ordering: default param values <user-supplied values < extra.
- Parameters
- extradict, optional
extra param values
- Returns
- dict
merged param map
- getInputCol()#
Gets the value of inputCol or its default value.
- getOrDefault(param)#
Gets the value of a param in the user-supplied param map or itsdefault value. Raises an error if neither is set.
- getOutputCol()#
Gets the value of outputCol or its default value.
- getParam(paramName)#
Gets a param by its name.
- hasDefault(param)#
Checks whether a param has a default value.
- hasParam(paramName)#
Tests whether this instance contains a param with a given(string) name.
- isDefined(param)#
Checks whether a param is explicitly set by user or hasa default value.
- isSet(param)#
Checks whether a param is explicitly set by user.
- classmethodload(path)#
Reads an ML instance from the input path, a shortcut ofread().load(path).
- classmethodread()#
Returns an MLReader instance for this class.
- save(path)#
Save this ML instance to the given path, a shortcut of ‘write().save(path)’.
- set(param,value)#
Sets a parameter in the embedded param map.
- setParams(self,\*,p=2.0,inputCol=None,outputCol=None)[source]#
Sets params for this Normalizer.
New in version 1.4.0.
- transform(dataset,params=None)#
Transforms the input dataset with optional parameters.
New in version 1.3.0.
- Parameters
- dataset
pyspark.sql.DataFrame input dataset
- paramsdict, optional
an optional param map that overrides embedded params.
- dataset
- Returns
pyspark.sql.DataFrametransformed dataset
- write()#
Returns an MLWriter instance for this ML instance.
Attributes Documentation
- inputCol=Param(parent='undefined',name='inputCol',doc='inputcolumnname.')#
- outputCol=Param(parent='undefined',name='outputCol',doc='outputcolumnname.')#
- p=Param(parent='undefined',name='p',doc='thepnormvalue.')#
- params#
Returns all params ordered by name. The default implementationuses
dir()to get all attributes of typeParam.
- uid#
A unique id for the object.