Spark Command: /opt/java/openjdk/bin/java -cp /opt/spark/conf:/opt/spark/jars/slf4j-api-2.0.17.jar:/opt/spark/jars/* -Xmx1g -XX:+IgnoreUnrecognizedVMOptions --add-modules=jdk.incubator.vector --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/jdk.internal.ref=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED --add-opens=java.base/sun.security.action=ALL-UNNAMED --add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED -Djdk.reflect.useDirectMethodHandle=false -Dio.netty.tryReflectionSetAccessible=true -Dio.netty.allocator.type=pooled -Dio.netty.handler.ssl.defaultEndpointVerificationAlgorithm=NONE --enable-native-access=ALL-UNNAMED org.apache.spark.deploy.worker.Worker --webui-port 8081 --webui-port 7079 spark://0.0.0.0:7077
========================================
WARNING: Using incubator modules: jdk.incubator.vector
Using Spark's default log4j profile: org/apache/spark/log4j2-defaults.properties
25/12/17 11:14:21 INFO Worker: Started daemon with process name: 98@spark-single-99fbfc65f-qj6t4
25/12/17 11:14:21 INFO SignalUtils: Registering signal handler for TERM
25/12/17 11:14:21 INFO SignalUtils: Registering signal handler for HUP
25/12/17 11:14:21 INFO SignalUtils: Registering signal handler for INT
25/12/17 11:14:21 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Using Spark's default log4j profile: org/apache/spark/log4j2-defaults.properties
25/12/17 11:14:21 INFO SecurityManager: Changing view acls to: spark
25/12/17 11:14:21 INFO SecurityManager: Changing modify acls to: spark
25/12/17 11:14:21 INFO SecurityManager: Changing view acls groups to: spark
25/12/17 11:14:21 INFO SecurityManager: Changing modify acls groups to: spark
25/12/17 11:14:21 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: spark groups with view permissions: EMPTY; users with modify permissions: spark; groups with modify permissions: EMPTY; RPC SSL disabled
25/12/17 11:14:21 INFO Utils: Successfully started service 'sparkWorker' on port 37813.
25/12/17 11:14:21 INFO Worker: Worker decommissioning not enabled.
25/12/17 11:14:21 INFO Worker: Starting Spark worker 10.42.0.101:37813 with 2 cores, 2.0 GiB RAM
25/12/17 11:14:21 INFO Worker: Running Spark version 4.1.0-preview4
25/12/17 11:14:21 INFO Worker: Spark home: /opt/spark
25/12/17 11:14:21 INFO ResourceUtils: ==============================================================
25/12/17 11:14:21 INFO ResourceUtils: No custom resources configured for spark.worker.
25/12/17 11:14:21 INFO ResourceUtils: ==============================================================
25/12/17 11:14:21 INFO JettyUtils: Start Jetty 0.0.0.0:7079 for WorkerUI
25/12/17 11:14:21 INFO Utils: Successfully started service 'WorkerUI' on port 7079.
25/12/17 11:14:21 INFO WorkerWebUI: Bound WorkerWebUI to 0.0.0.0, and started at http://spark-single-99fbfc65f-qj6t4:7079
25/12/17 11:14:21 INFO Worker: Connecting to master 0.0.0.0:7077...
25/12/17 11:14:21 INFO TransportClientFactory: Successfully created connection to /0.0.0.0:7077 after 11 ms (0 ms spent in bootstraps)
25/12/17 11:14:22 INFO Worker: Successfully registered with master spark://0.0.0.0:7077
25/12/17 11:14:22 INFO Worker: Worker cleanup enabled; old application directories will be deleted in: /opt/spark/work
25/12/17 11:14:22 INFO Worker: Asked to launch executor app-20251217111421-0000/0 for Kotlin Spark Example
25/12/17 11:14:22 INFO SecurityManager: Changing view acls to: spark
25/12/17 11:14:22 INFO SecurityManager: Changing modify acls to: spark
25/12/17 11:14:22 INFO SecurityManager: Changing view acls groups to: spark
25/12/17 11:14:22 INFO SecurityManager: Changing modify acls groups to: spark
25/12/17 11:14:22 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: spark groups with view permissions: EMPTY; users with modify permissions: spark; groups with modify permissions: EMPTY; RPC SSL disabled
25/12/17 11:14:22 INFO ExecutorRunner: Launch command: "/opt/java/openjdk/bin/java" "-cp" "/opt/spark/conf:/opt/spark/jars/slf4j-api-2.0.17.jar:/opt/spark/jars/*" "-Xmx1024M" "-Dspark.driver.port=56309" "-Djava.net.preferIPv6Addresses=false" "-XX:+IgnoreUnrecognizedVMOptions" "--add-modules=jdk.incubator.vector" "--add-opens=java.base/java.lang=ALL-UNNAMED" "--add-opens=java.base/java.lang.invoke=ALL-UNNAMED" "--add-opens=java.base/java.lang.reflect=ALL-UNNAMED" "--add-opens=java.base/java.io=ALL-UNNAMED" "--add-opens=java.base/java.net=ALL-UNNAMED" "--add-opens=java.base/java.nio=ALL-UNNAMED" "--add-opens=java.base/java.util=ALL-UNNAMED" "--add-opens=java.base/java.util.concurrent=ALL-UNNAMED" "--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED" "--add-opens=java.base/jdk.internal.ref=ALL-UNNAMED" "--add-opens=java.base/sun.nio.ch=ALL-UNNAMED" "--add-opens=java.base/sun.nio.cs=ALL-UNNAMED" "--add-opens=java.base/sun.security.action=ALL-UNNAMED" "--add-opens=java.base/sun.util.calendar=ALL-UNNAMED" "--add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED" "-Djdk.reflect.useDirectMethodHandle=false" "-Dio.netty.tryReflectionSetAccessible=true" "-Dio.netty.allocator.type=pooled" "-Dio.netty.handler.ssl.defaultEndpointVerificationAlgorithm=NONE" "--enable-native-access=ALL-UNNAMED" "org.apache.spark.executor.CoarseGrainedExecutorBackend" "--driver-url" "spark://CoarseGrainedScheduler@mac:56309" "--executor-id" "0" "--hostname" "10.42.0.101" "--cores" "2" "--app-id" "app-20251217111421-0000" "--worker-url" "spark://Worker@10.42.0.101:37813" "--resourceProfileId" "0"
25/12/17 11:14:23 INFO Worker: Executor app-20251217111421-0000/0 finished with state EXITED message Command exited with code 1 exitStatus 1
25/12/17 11:14:23 INFO ExternalShuffleBlockResolver: Clean up non-shuffle and non-RDD files associated with the finished executor 0
25/12/17 11:14:23 INFO ExternalShuffleBlockResolver: Executor is not registered (appId=app-20251217111421-0000, execId=0)
25/12/17 11:14:23 INFO Worker: Asked to launch executor app-20251217111421-0000/1 for Kotlin Spark Example
25/12/17 11:14:23 INFO SecurityManager: Changing view acls to: spark
25/12/17 11:14:23 INFO SecurityManager: Changing modify acls to: spark
25/12/17 11:14:23 INFO SecurityManager: Changing view acls groups to: spark
25/12/17 11:14:23 INFO SecurityManager: Changing modify acls groups to: spark
25/12/17 11:14:23 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: spark groups with view permissions: EMPTY; users with modify permissions: spark; groups with modify permissions: EMPTY; RPC SSL disabled
25/12/17 11:14:23 INFO ExecutorRunner: Launch command: "/opt/java/openjdk/bin/java" "-cp" "/opt/spark/conf:/opt/spark/jars/slf4j-api-2.0.17.jar:/opt/spark/jars/*" "-Xmx1024M" "-Dspark.driver.port=56309" "-Djava.net.preferIPv6Addresses=false" "-XX:+IgnoreUnrecognizedVMOptions" "--add-modules=jdk.incubator.vector" "--add-opens=java.base/java.lang=ALL-UNNAMED" "--add-opens=java.base/java.lang.invoke=ALL-UNNAMED" "--add-opens=java.base/java.lang.reflect=ALL-UNNAMED" "--add-opens=java.base/java.io=ALL-UNNAMED" "--add-opens=java.base/java.net=ALL-UNNAMED" "--add-opens=java.base/java.nio=ALL-UNNAMED" "--add-opens=java.base/java.util=ALL-UNNAMED" "--add-opens=java.base/java.util.concurrent=ALL-UNNAMED" "--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED" "--add-opens=java.base/jdk.internal.ref=ALL-UNNAMED" "--add-opens=java.base/sun.nio.ch=ALL-UNNAMED" "--add-opens=java.base/sun.nio.cs=ALL-UNNAMED" "--add-opens=java.base/sun.security.action=ALL-UNNAMED" "--add-opens=java.base/sun.util.calendar=ALL-UNNAMED" "--add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED" "-Djdk.reflect.useDirectMethodHandle=false" "-Dio.netty.tryReflectionSetAccessible=true" "-Dio.netty.allocator.type=pooled" "-Dio.netty.handler.ssl.defaultEndpointVerificationAlgorithm=NONE" "--enable-native-access=ALL-UNNAMED" "org.apache.spark.executor.CoarseGrainedExecutorBackend" "--driver-url" "spark://CoarseGrainedScheduler@mac:56309" "--executor-id" "1" "--hostname" "10.42.0.101" "--cores" "2" "--app-id" "app-20251217111421-0000" "--worker-url" "spark://Worker@10.42.0.101:37813" "--resourceProfileId" "0"
25/12/17 11:14:24 INFO Worker: Executor app-20251217111421-0000/1 finished with state EXITED message Command exited with code 1 exitStatus 1
25/12/17 11:14:24 INFO ExternalShuffleBlockResolver: Clean up non-shuffle and non-RDD files associated with the finished executor 1
25/12/17 11:14:24 INFO ExternalShuffleBlockResolver: Executor is not registered (appId=app-20251217111421-0000, execId=1)
25/12/17 11:14:24 INFO Worker: Asked to launch executor app-20251217111421-0000/2 for Kotlin Spark Example
25/12/17 11:14:24 INFO SecurityManager: Changing view acls to: spark
25/12/17 11:14:24 INFO SecurityManager: Changing modify acls to: spark
25/12/17 11:14:24 INFO SecurityManager: Changing view acls groups to: spark
25/12/17 11:14:24 INFO SecurityManager: Changing modify acls groups to: spark
25/12/17 11:14:24 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: spark groups with view permissions: EMPTY; users with modify permissions: spark; groups with modify permissions: EMPTY; RPC SSL disabled
25/12/17 11:14:24 INFO ExecutorRunner: Launch command: "/opt/java/openjdk/bin/java" "-cp" "/opt/spark/conf:/opt/spark/jars/slf4j-api-2.0.17.jar:/opt/spark/jars/*" "-Xmx1024M" "-Dspark.driver.port=56309" "-Djava.net.preferIPv6Addresses=false" "-XX:+IgnoreUnrecognizedVMOptions" "--add-modules=jdk.incubator.vector" "--add-opens=java.base/java.lang=ALL-UNNAMED" "--add-opens=java.base/java.lang.invoke=ALL-UNNAMED" "--add-opens=java.base/java.lang.reflect=ALL-UNNAMED" "--add-opens=java.base/java.io=ALL-UNNAMED" "--add-opens=java.base/java.net=ALL-UNNAMED" "--add-opens=java.base/java.nio=ALL-UNNAMED" "--add-opens=java.base/java.util=ALL-UNNAMED" "--add-opens=java.base/java.util.concurrent=ALL-UNNAMED" "--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED" "--add-opens=java.base/jdk.internal.ref=ALL-UNNAMED" "--add-opens=java.base/sun.nio.ch=ALL-UNNAMED" "--add-opens=java.base/sun.nio.cs=ALL-UNNAMED" "--add-opens=java.base/sun.security.action=ALL-UNNAMED" "--add-opens=java.base/sun.util.calendar=ALL-UNNAMED" "--add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED" "-Djdk.reflect.useDirectMethodHandle=false" "-Dio.netty.tryReflectionSetAccessible=true" "-Dio.netty.allocator.type=pooled" "-Dio.netty.handler.ssl.defaultEndpointVerificationAlgorithm=NONE" "--enable-native-access=ALL-UNNAMED" "org.apache.spark.executor.CoarseGrainedExecutorBackend" "--driver-url" "spark://CoarseGrainedScheduler@mac:56309" "--executor-id" "2" "--hostname" "10.42.0.101" "--cores" "2" "--app-id" "app-20251217111421-0000" "--worker-url" "spark://Worker@10.42.0.101:37813" "--resourceProfileId" "0"
25/12/17 11:14:25 INFO Worker: Executor app-20251217111421-0000/2 finished with state EXITED message Command exited with code 1 exitStatus 1
25/12/17 11:14:25 INFO ExternalShuffleBlockResolver: Clean up non-shuffle and non-RDD files associated with the finished executor 2
25/12/17 11:14:25 INFO ExternalShuffleBlockResolver: Executor is not registered (appId=app-20251217111421-0000, execId=2)
25/12/17 11:14:25 INFO Worker: Asked to launch executor app-20251217111421-0000/3 for Kotlin Spark Example
25/12/17 11:14:25 INFO SecurityManager: Changing view acls to: spark
25/12/17 11:14:25 INFO SecurityManager: Changing modify acls to: spark
25/12/17 11:14:25 INFO SecurityManager: Changing view acls groups to: spark
25/12/17 11:14:25 INFO SecurityManager: Changing modify acls groups to: spark
25/12/17 11:14:25 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: spark groups with view permissions: EMPTY; users with modify permissions: spark; groups with modify permissions: EMPTY; RPC SSL disabled
25/12/17 11:14:25 INFO ExecutorRunner: Launch command: "/opt/java/openjdk/bin/java" "-cp" "/opt/spark/conf:/opt/spark/jars/slf4j-api-2.0.17.jar:/opt/spark/jars/*" "-Xmx1024M" "-Dspark.driver.port=56309" "-Djava.net.preferIPv6Addresses=false" "-XX:+IgnoreUnrecognizedVMOptions" "--add-modules=jdk.incubator.vector" "--add-opens=java.base/java.lang=ALL-UNNAMED" "--add-opens=java.base/java.lang.invoke=ALL-UNNAMED" "--add-opens=java.base/java.lang.reflect=ALL-UNNAMED" "--add-opens=java.base/java.io=ALL-UNNAMED" "--add-opens=java.base/java.net=ALL-UNNAMED" "--add-opens=java.base/java.nio=ALL-UNNAMED" "--add-opens=java.base/java.util=ALL-UNNAMED" "--add-opens=java.base/java.util.concurrent=ALL-UNNAMED" "--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED" "--add-opens=java.base/jdk.internal.ref=ALL-UNNAMED" "--add-opens=java.base/sun.nio.ch=ALL-UNNAMED" "--add-opens=java.base/sun.nio.cs=ALL-UNNAMED" "--add-opens=java.base/sun.security.action=ALL-UNNAMED" "--add-opens=java.base/sun.util.calendar=ALL-UNNAMED" "--add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED" "-Djdk.reflect.useDirectMethodHandle=false" "-Dio.netty.tryReflectionSetAccessible=true" "-Dio.netty.allocator.type=pooled" "-Dio.netty.handler.ssl.defaultEndpointVerificationAlgorithm=NONE" "--enable-native-access=ALL-UNNAMED" "org.apache.spark.executor.CoarseGrainedExecutorBackend" "--driver-url" "spark://CoarseGrainedScheduler@mac:56309" "--executor-id" "3" "--hostname" "10.42.0.101" "--cores" "2" "--app-id" "app-20251217111421-0000" "--worker-url" "spark://Worker@10.42.0.101:37813" "--resourceProfileId" "0"
25/12/17 11:14:26 INFO Worker: Executor app-20251217111421-0000/3 finished with state EXITED message Command exited with code 1 exitStatus 1
25/12/17 11:14:26 INFO ExternalShuffleBlockResolver: Clean up non-shuffle and non-RDD files associated with the finished executor 3
25/12/17 11:14:26 INFO ExternalShuffleBlockResolver: Executor is not registered (appId=app-20251217111421-0000, execId=3)
25/12/17 11:14:26 INFO Worker: Asked to launch executor app-20251217111421-0000/4 for Kotlin Spark Example
25/12/17 11:14:26 INFO SecurityManager: Changing view acls to: spark
25/12/17 11:14:26 INFO SecurityManager: Changing modify acls to: spark
25/12/17 11:14:26 INFO SecurityManager: Changing view acls groups to: spark
25/12/17 11:14:26 INFO SecurityManager: Changing modify acls groups to: spark
25/12/17 11:14:26 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: spark groups with view permissions: EMPTY; users with modify permissions: spark; groups with modify permissions: EMPTY; RPC SSL disabled
25/12/17 11:14:26 INFO ExecutorRunner: Launch command: "/opt/java/openjdk/bin/java" "-cp" "/opt/spark/conf:/opt/spark/jars/slf4j-api-2.0.17.jar:/opt/spark/jars/*" "-Xmx1024M" "-Dspark.driver.port=56309" "-Djava.net.preferIPv6Addresses=false" "-XX:+IgnoreUnrecognizedVMOptions" "--add-modules=jdk.incubator.vector" "--add-opens=java.base/java.lang=ALL-UNNAMED" "--add-opens=java.base/java.lang.invoke=ALL-UNNAMED" "--add-opens=java.base/java.lang.reflect=ALL-UNNAMED" "--add-opens=java.base/java.io=ALL-UNNAMED" "--add-opens=java.base/java.net=ALL-UNNAMED" "--add-opens=java.base/java.nio=ALL-UNNAMED" "--add-opens=java.base/java.util=ALL-UNNAMED" "--add-opens=java.base/java.util.concurrent=ALL-UNNAMED" "--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED" "--add-opens=java.base/jdk.internal.ref=ALL-UNNAMED" "--add-opens=java.base/sun.nio.ch=ALL-UNNAMED" "--add-opens=java.base/sun.nio.cs=ALL-UNNAMED" "--add-opens=java.base/sun.security.action=ALL-UNNAMED" "--add-opens=java.base/sun.util.calendar=ALL-UNNAMED" "--add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED" "-Djdk.reflect.useDirectMethodHandle=false" "-Dio.netty.tryReflectionSetAccessible=true" "-Dio.netty.allocator.type=pooled" "-Dio.netty.handler.ssl.defaultEndpointVerificationAlgorithm=NONE" "--enable-native-access=ALL-UNNAMED" "org.apache.spark.executor.CoarseGrainedExecutorBackend" "--driver-url" "spark://CoarseGrainedScheduler@mac:56309" "--executor-id" "4" "--hostname" "10.42.0.101" "--cores" "2" "--app-id" "app-20251217111421-0000" "--worker-url" "spark://Worker@10.42.0.101:37813" "--resourceProfileId" "0"
25/12/17 11:14:27 INFO Worker: Asked to kill executor app-20251217111421-0000/4
25/12/17 11:14:27 INFO ExecutorRunner: Runner thread for executor app-20251217111421-0000/4 interrupted
25/12/17 11:14:27 INFO ExecutorRunner: Killing process!
25/12/17 11:14:27 INFO Worker: Executor app-20251217111421-0000/4 finished with state KILLED message Runner thread for executor app-20251217111421-0000/4 interrupted exitStatus 143
25/12/17 11:14:27 INFO ExternalShuffleBlockResolver: Clean up non-shuffle and non-RDD files associated with the finished executor 4
25/12/17 11:14:27 INFO ExternalShuffleBlockResolver: Executor is not registered (appId=app-20251217111421-0000, execId=4)
25/12/17 11:14:27 INFO ExternalShuffleBlockResolver: Application app-20251217111421-0000 removed, cleanupLocalDirs = true
25/12/17 11:14:27 INFO Worker: Cleaning up local directories for application app-20251217111421-0000