spark_connect(master, spark_home = Sys.getenv("SPARK_HOME"), method = c("shell"), app_name = "sparklyr", version = NULL, hadoop_version = NULL, config = spark_config(), extensions = sparklyr::registered_extensions())
- Spark cluster url to connect to. Use
"local" to connect to a local
instance of Spark installed via
- Spark home directory (defaults to SPARK_HOME environment variable).
SPARK_HOME is defined it will be always be used unless the
paramater is specified to force the use of a locally installed version.
- of connecting to spark (currently only "shell" is supported, additional
methods may come later)
- Application name to be used while running in the Spark cluster
- Version of Spark (only applicable for local master)
- Version of Hadoop (only applicable for local master)
- Configuration for connection (see
spark_config for details).
- Extension packages to enable for this connection. By default will
enable all packages that previously called
Connection to Spark local instance or remote cluster
Connect to Spark