Spark操作员停止使用Kubernetes

Spark Operator工作正常,但突然停止工作,并出现以下错误。 kubernetes版本或spark-operator版本没有变化。

Exception in thread "kubernetes-dispatcher-0" java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFuturetask@1ca8f413 rejected from java.util.concurrent.ScheduledThreadPoolExecutor@23fb84b3[Terminated,pool size = 0,active threads = 0,queued tasks = 0,completed tasks = 0]
    at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
    at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
    at java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:326)
    at java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:533)
    at java.util.concurrent.ScheduledThreadPoolExecutor.submit(ScheduledThreadPoolExecutor.java:632)
    at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678)
    at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionmanager.scheduleReconnect(WatchConnectionmanager.java:303)
    at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionmanager.access$800(WatchConnectionmanager.java:48)
    at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionmanager$2.onFailure(WatchConnectionmanager.java:216)
    at okhttp3.internal.ws.RealWebSocket.failWebSocket(RealWebSocket.java:543)
    at okhttp3.internal.ws.RealWebSocket$2.onResponse(RealWebSocket.java:185)
    at okhttp3.RealCall$AsyncCall.execute(RealCall.java:141)
    at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Exception in thread "main" io.fabric8.kubernetes.client.KubernetesClientException: 
    at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionmanager$2.onFailure(WatchConnectionmanager.java:201)
    at okhttp3.internal.ws.RealWebSocket.failWebSocket(RealWebSocket.java:543)
    at okhttp3.internal.ws.RealWebSocket$2.onResponse(RealWebSocket.java:185)
    at okhttp3.RealCall$AsyncCall.execute(RealCall.java:141)
    at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
ugrgadd1 回答:Spark操作员停止使用Kubernetes

有同样的问题(大约11月5日)。这可能是由于GKE主节点和节点池gke release notes的自动升级引起的。

更改kubernetes客户端版本可以达到目的:我删除了$SPARK_HOME/jars/kubernetes-client-4.1.2.jar并替换为$SPARK_HOME/jars/kubernetes-client-4.4.2.jarlink来下载新客户端)

link of issue opened in GoogleCloudPlatform/spark-on-k8s-operator project

中找到了解决方案
本文链接:https://www.f2er.com/3161363.html

大家都在问