1
私はスパークをインストールし、チュートリアルの所与のコマンドを実行するが、次のエラーを取得しよう -列オブジェクト呼び出し可能ではないスパーク
https://spark.apache.org/docs/latest/quick-start.html
P-MBP:spark-2.0.2-bin-hadoop2.4 prem$ ./bin/pyspark
Python 2.7.13 (default, Apr 4 2017, 08:44:49)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel).
17/09/12 17:26:53 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Welcome to
____ __
/__/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/__/.__/\_,_/_/ /_/\_\ version 2.0.2
/_/
Using Python version 2.7.13 (default, Apr 4 2017 08:44:49)
SparkSession available as 'spark'.
>>> textFile = spark.read.text("README.md")
>>> textFile.count()
99
>>> textFile.first()
Row(value=u'# Apache Spark')
>>> linesWithSpark = textFile.filter(textFile.value.contains("Spark"))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'Column' object is not callable
>>> dir(textFile.value)
['__add__', '__and__', '__bool__', '__class__', '__contains__', '__delattr__', '__dict__', '__div__', '__doc__', '__eq__', '__format__', '__ge__', '__getattr__', '__getattribute__', '__getitem__', '__getslice__', '__gt__', '__hash__', '__init__', '__invert__', '__iter__', '__le__', '__lt__', '__mod__', '__module__', '__mul__', '__ne__', '__neg__', '__new__', '__nonzero__', '__or__', '__pow__', '__radd__', '__rand__', '__rdiv__', '__reduce__', '__reduce_ex__', '__repr__', '__rmod__', '__rmul__', '__ror__', '__rpow__', '__rsub__', '__rtruediv__', '__setattr__', '__sizeof__', '__str__', '__sub__', '__subclasshook__', '__truediv__', '__weakref__', '_jc', 'alias', 'asc', 'astype', 'between', 'bitwiseAND', 'bitwiseOR', 'bitwiseXOR', 'cast', 'desc', 'endswith', 'getField', 'getItem', 'isNotNull', 'isNull', 'isin', 'like', 'name', 'otherwise', 'over', 'rlike', 'startswith', 'substr', 'when']