TensorFrames lets you manipulate Apache Spark’s DataFrames with TensorFlow programs
Note: TensorFrames is deprecated. You can use
pandas UDFinstead.
Experimental TensorFlow binding for Scala and
Apache Spark.
TensorFrames (TensorFlow on Spark DataFrames) lets you manipulate Apache Spark’s DataFrames with
TensorFlow programs.
This package is experimental and is provided as a technical preview only. While the
interfaces are all implemented and working, there are still some areas of low performance.
Supported platforms:
This package only officially supports linux 64bit platforms as a target.
Contributions are welcome for other platforms.
See the file project/Dependencies.scala for adding your own platform.
Officially TensorFrames supports Spark 2.4+ and Scala 2.11.
See the user guide for
extensive information about the API.
For questions, see the TensorFrames mailing list.
TensorFrames is available as a
Spark package.
Requirements
A working version of Apache Spark (2.4 or greater)
Java 8+
(Optional) python 2.7+/3.6+ if you want to use the python