Skip to content

Commit 72fa842

Browse files
authored
[Flink] Make Flink 2.x job submission work on REMOTE mode (#4500)
* [Flink] Fix Flink SQL job submission failures on the client path Six defects on the path that turns a saved Flink SQL application into a submitted job. Each was found by submitting a real Flink SQL job to a standalone cluster and fixing whatever failed next; they are independent of each other but all sit on this one path. 1. ClassLoaderUtils.runAsClassLoader restored the context classloader captured in a static field when the class was first initialized — whichever thread happened to load it — instead of the one the calling thread had on entry. On pooled threads that silently replaces an unrelated thread's context classloader. 2. FlinkClientTrait.getCustomCommandLines and RemoteClient.getStandAloneClusterDescriptor call into Flink classes bound to the Flink version bundled with this module, while the calling thread's context classloader is FlinkShimsProxy's target-version shims classloader. Their internal ServiceLoader lookups therefore resolved providers from a different Flink version than the interfaces bundled here, failing with ServiceConfigurationError ("not a subtype"). Both call sites now run under their own class's classloader. Closes #4483. 3. The build-response getters (workspacePath, pass, shadedJarPath, flinkBaseImage, mainJarPath, extraLibJarPaths, flinkImageTag, podTemplatePaths, dockerInnerMainJarPath) do not follow JavaBean getter naming and carried no @JsonProperty, so Jackson silently skipped them: every build result persisted to t_flink_app's buildResultJson lost its paths, and only pass survived — by the coincidence that its field default is already true. A later submit then read back shadedJarPath == null and failed with an NPE, an "entry point class not found", or "flinkJobJar is null", depending on which downstream path consumed it. 4. SubmitRequest.userJarFile() passed shadedJarPath() straight to new File(...), which throws NPE when it is legitimately null. 5. streampark-console-service declared a compile dependency on streampark-flink-shims-base but not on streampark-flink-shims-base-v2, so FlinkTableInitializerV2 never reached the console's lib/ and every Flink 2.x SQL job failed with NoClassDefFoundError. Flink 1.x was unaffected, which is why this went unnoticed. 6. PackagedProgram's setUserClassPaths, disabled wholesale for #3761, is re-enabled for FLINK_SQL jobs only, so a SQL job's connector jars reach the client classpath. Verified against a real cluster not to reproduce the ClassCastException #3761 describes, and it leaves every other job type on the existing behaviour. * [Flink] Make Flink 2.x job submission work on REMOTE mode Submitting a Flink SQL job to a Flink 2.x cluster failed with a NoClassDefFoundError long before reaching the cluster. Flink 1.x was unaffected, which is why this went unnoticed. Five independent causes, each of which only becomes visible after the previous one is fixed: 1. FlinkShimsProxy stopped putting the version-specific shims jar into the shims classloader. It matches on a name shaped "streampark-flink-shims_flink-<ver>_<scala>", which those artifacts carried until e770d2e renamed them without the Scala suffix. Both spellings are accepted now. 2. The same rename silenced the rule that pulls in the rest of StreamPark's Flink jars ("has a _<scala> suffix"), so the client stack was loaded by the console's own classloader and resolved org.apache.flink.* from the console's fixed baseline Flink instead of the target version's jars. That is the actual mechanism behind #4483: the ServiceLoader mismatch it reports is what a half-populated shims classloader looks like from the outside. 3. shims-base and shims-base-v2 share twelve class names — v2 redeclares them for Flink 2.x and inherits the rest — but both were added to every shims classloader regardless of the target version, in directory listing order. Which Flink version a class had been compiled for was therefore decided by the filesystem. A 1.x target no longer sees v2 at all, and a 2.x target gets v2 ahead of the base. 4. SavepointConfigOptions was removed in Flink 2.x, and Configuration's typed accessors (getBoolean/setBoolean/getInteger over a ConfigOption) went with it. Since this module is compiled once against a single baseline but submits to whichever version the user registered, both are now addressed portably: the savepoint options are declared from their keys, which are byte-identical across every supported version, and the generic get/set replace the typed accessors. 5. ClusterClient#submitJob widened its parameter from JobGraph to ExecutionPlan in 2.x. The instance satisfies either signature, only the declared type moved, so the call is made reflectively. A FLINK_SQL program's classloader is also told to resolve org.apache.streampark.* parent-first. The fat jar bundles whichever shims it was built against, while the parent is the shims classloader for the version actually registered; loading both ends in a LinkageError as soon as one references the other. Verified against real standalone clusters by driving StreamPark's own submission path out-of-process: a Flink SQL job now submits and reaches FINISHED on Flink 2.2.1, and the same job on Flink 1.20.4 — which worked before this change — still does. Not addressed: LocalClient and KubernetesNativeSessionClient use the same removed Configuration accessors and will fail the same way on Flink 2.x. Neither is reachable in the environment this was verified in, so they are left for a change that can be tested.
1 parent 9ddda84 commit 72fa842

12 files changed

Lines changed: 218 additions & 30 deletions

File tree

streampark-common/src/main/java/org/apache/streampark/common/util/ClassLoaderUtils.java

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,11 +39,12 @@ private ClassLoaderUtils() {
3939
}
4040

4141
public static <R> R runAsClassLoader(ClassLoader targetClassLoader, Supplier<R> supplier) {
42+
ClassLoader previousClassLoader = Thread.currentThread().getContextClassLoader();
4243
try {
4344
Thread.currentThread().setContextClassLoader(targetClassLoader);
4445
return supplier.get();
4546
} finally {
46-
Thread.currentThread().setContextClassLoader(ORIGINAL_CLASS_LOADER);
47+
Thread.currentThread().setContextClassLoader(previousClassLoader);
4748
}
4849
}
4950

streampark-console/streampark-console-service/pom.xml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -372,6 +372,12 @@
372372
<version>${project.version}</version>
373373
</dependency>
374374

375+
<dependency>
376+
<groupId>org.apache.streampark</groupId>
377+
<artifactId>streampark-flink-shims-base-v2</artifactId>
378+
<version>${project.version}</version>
379+
</dependency>
380+
375381
<!-- Ensure all Flink shims are built when console-service is built with -am -->
376382
<dependency>
377383
<groupId>org.apache.streampark</groupId>

streampark-flink/streampark-flink-client/streampark-flink-client-api/src/main/java/org/apache/streampark/flink/client/bean/SubmitRequest.java

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,14 +30,14 @@
3030
import org.apache.streampark.common.util.DeflaterUtils;
3131
import org.apache.streampark.common.util.HdfsUtils;
3232
import org.apache.streampark.common.util.PropertiesUtils;
33+
import org.apache.streampark.flink.client.conf.FlinkSavepointOptions;
3334
import org.apache.streampark.flink.packer.pipeline.BuildResult;
3435
import org.apache.streampark.flink.packer.pipeline.ShadedBuildResponse;
3536

3637
import org.apache.streampark.shaded.com.fasterxml.jackson.core.type.TypeReference;
3738
import org.apache.streampark.shaded.com.fasterxml.jackson.databind.ObjectMapper;
3839

3940
import org.apache.commons.collections.MapUtils;
40-
import org.apache.flink.runtime.jobgraph.SavepointConfigOptions;
4141
import org.apache.flink.runtime.jobgraph.SavepointRestoreSettings;
4242

4343
import javax.annotation.Nullable;
@@ -263,7 +263,7 @@ public String flinkSQL() {
263263
public boolean allowNonRestoredState() {
264264
if (allowNonRestoredState == null) {
265265
Object value =
266-
properties.get(SavepointConfigOptions.SAVEPOINT_IGNORE_UNCLAIMED_STATE.key());
266+
properties.get(FlinkSavepointOptions.SAVEPOINT_IGNORE_UNCLAIMED_STATE.key());
267267
if (value == null) {
268268
allowNonRestoredState = false;
269269
} else {
@@ -296,7 +296,8 @@ public File userJarFile() {
296296
} else {
297297
checkBuildResult();
298298
ShadedBuildResponse shadedBuildResult = buildResult.as(ShadedBuildResponse.class);
299-
userJarFile = new File(shadedBuildResult.shadedJarPath());
299+
String shadedJarPath = shadedBuildResult.shadedJarPath();
300+
userJarFile = shadedJarPath == null ? null : new File(shadedJarPath);
300301
}
301302
}
302303
return userJarFile;
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one or more
3+
* contributor license agreements. See the NOTICE file distributed with
4+
* this work for additional information regarding copyright ownership.
5+
* The ASF licenses this file to You under the Apache License, Version 2.0
6+
* (the "License"); you may not use this file except in compliance with
7+
* the License. You may obtain a copy of the License at
8+
*
9+
* http://www.apache.org/licenses/LICENSE-2.0
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS,
13+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*/
17+
18+
package org.apache.streampark.flink.client.conf;
19+
20+
import org.apache.flink.configuration.ConfigOption;
21+
import org.apache.flink.configuration.ConfigOptions;
22+
23+
/**
24+
* The savepoint restore options, declared here rather than taken from Flink.
25+
*
26+
* <p>Flink kept moving the class that declares them — {@code
27+
* org.apache.flink.runtime.jobgraph.SavepointConfigOptions} through 1.x, {@code
28+
* org.apache.flink.configuration.StateRecoveryOptions} in 2.x — while keeping the option *keys*
29+
* byte-identical across every version this project supports. Since this module is compiled once
30+
* against a single baseline Flink but submits to whichever version the user registered, referring
31+
* to either class binds the client to one version family and fails against the other with a
32+
* {@code NoClassDefFoundError} at submission time. Declaring the options from their keys sidesteps
33+
* that entirely: a {@code Configuration} is keyed by string, so a locally declared option addresses
34+
* exactly the same setting as Flink's own.
35+
*
36+
* <p>The keys are part of Flink's public configuration surface, so they are as stable as the user's
37+
* own {@code flink-conf.yaml} entries.
38+
*/
39+
public final class FlinkSavepointOptions {
40+
41+
/** Mirrors Flink's {@code execution.savepoint.path}. */
42+
public static final ConfigOption<String> SAVEPOINT_PATH =
43+
ConfigOptions.key("execution.savepoint.path")
44+
.stringType()
45+
.noDefaultValue()
46+
.withDescription("Path to a savepoint to restore the job from.");
47+
48+
/** Mirrors Flink's {@code execution.savepoint.ignore-unclaimed-state}. */
49+
public static final ConfigOption<Boolean> SAVEPOINT_IGNORE_UNCLAIMED_STATE =
50+
ConfigOptions.key("execution.savepoint.ignore-unclaimed-state")
51+
.booleanType()
52+
.defaultValue(false)
53+
.withDescription("Allow to skip savepoint state that cannot be restored.");
54+
55+
private FlinkSavepointOptions() {
56+
}
57+
}

streampark-flink/streampark-flink-client/streampark-flink-client-core/src/main/java/org/apache/streampark/flink/client/impl/RemoteClient.java

Lines changed: 20 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717

1818
package org.apache.streampark.flink.client.impl;
1919

20+
import org.apache.streampark.common.util.ClassLoaderUtils;
2021
import org.apache.streampark.flink.client.bean.CancelRequest;
2122
import org.apache.streampark.flink.client.bean.CancelResponse;
2223
import org.apache.streampark.flink.client.bean.SavepointRequestTrait;
@@ -165,13 +166,25 @@ private <O, R extends SavepointRequestTrait> O executeClientAction(
165166

166167
private Tuple2<StandaloneClusterId, StandaloneClusterDescriptor> getStandAloneClusterDescriptor(
167168
Configuration flinkConfig) {
168-
DefaultClusterClientServiceLoader serviceLoader = new DefaultClusterClientServiceLoader();
169-
ClusterClientFactory<StandaloneClusterId> clientFactory =
170-
serviceLoader.getClusterClientFactory(flinkConfig);
171-
StandaloneClusterId standaloneClusterId = clientFactory.getClusterId(flinkConfig);
172-
StandaloneClusterDescriptor standaloneClusterDescriptor =
173-
(StandaloneClusterDescriptor) clientFactory.createClusterDescriptor(flinkConfig);
174-
return new Tuple2<>(standaloneClusterId, standaloneClusterDescriptor);
169+
// DefaultClusterClientServiceLoader is bound to the Flink version bundled with this module
170+
// (loaded by this class's own classloader), but the calling thread's context classloader may
171+
// currently be a target-version shims classloader (see FlinkShimsProxy). Its internal
172+
// ServiceLoader.load(ClusterClientFactory.class) resolves providers via the context
173+
// classloader, so leaving it as the shims classloader here would load a ClusterClientFactory
174+
// implementation from a different Flink version than the interface bundled here, throwing
175+
// ServiceConfigurationError ("not a subtype"). Force it back to this class's own classloader
176+
// for the duration of this call.
177+
return ClassLoaderUtils.runAsClassLoader(
178+
RemoteClient.class.getClassLoader(),
179+
() -> {
180+
DefaultClusterClientServiceLoader serviceLoader = new DefaultClusterClientServiceLoader();
181+
ClusterClientFactory<StandaloneClusterId> clientFactory =
182+
serviceLoader.getClusterClientFactory(flinkConfig);
183+
StandaloneClusterId standaloneClusterId = clientFactory.getClusterId(flinkConfig);
184+
StandaloneClusterDescriptor standaloneClusterDescriptor =
185+
(StandaloneClusterDescriptor) clientFactory.createClusterDescriptor(flinkConfig);
186+
return new Tuple2<>(standaloneClusterId, standaloneClusterDescriptor);
187+
});
175188
}
176189

177190
@FunctionalInterface

streampark-flink/streampark-flink-client/streampark-flink-client-core/src/main/java/org/apache/streampark/flink/client/tool/FlinkSessionSubmitHelper.java

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -20,14 +20,14 @@
2020
import org.apache.streampark.common.util.AssertUtils;
2121
import org.apache.streampark.common.util.JsonUtils;
2222
import org.apache.streampark.common.util.LoggerSupport;
23+
import org.apache.streampark.flink.client.conf.FlinkSavepointOptions;
2324
import org.apache.streampark.flink.kubernetes.KubernetesRetriever;
2425

2526
import org.apache.streampark.shaded.com.fasterxml.jackson.databind.JsonNode;
2627

2728
import org.apache.flink.client.deployment.application.ApplicationConfiguration;
2829
import org.apache.flink.configuration.Configuration;
2930
import org.apache.flink.configuration.CoreOptions;
30-
import org.apache.flink.runtime.jobgraph.SavepointConfigOptions;
3131
import org.apache.hc.client5.http.entity.mime.MultipartEntityBuilder;
3232
import org.apache.hc.client5.http.fluent.Request;
3333
import org.apache.hc.core5.http.ContentType;
@@ -161,9 +161,9 @@ class JarRunRequest {
161161
List<String> args = flinkConf.get(ApplicationConfiguration.APPLICATION_ARGS);
162162
this.programArgs = args == null ? null : String.join(" ", args);
163163
this.parallelism = String.valueOf(flinkConf.get(CoreOptions.DEFAULT_PARALLELISM));
164-
this.savepointPath = flinkConf.get(SavepointConfigOptions.SAVEPOINT_PATH);
164+
this.savepointPath = flinkConf.get(FlinkSavepointOptions.SAVEPOINT_PATH);
165165
this.allowNonRestoredState =
166-
flinkConf.getBoolean(SavepointConfigOptions.SAVEPOINT_IGNORE_UNCLAIMED_STATE);
166+
flinkConf.get(FlinkSavepointOptions.SAVEPOINT_IGNORE_UNCLAIMED_STATE);
167167
}
168168

169169
public String getEntryClass() {

streampark-flink/streampark-flink-client/streampark-flink-client-core/src/main/java/org/apache/streampark/flink/client/trait/FlinkClientTrait.java

Lines changed: 69 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@
2626
import org.apache.streampark.common.enums.FlinkRestoreMode;
2727
import org.apache.streampark.common.fs.FsOperator;
2828
import org.apache.streampark.common.util.AssertUtils;
29+
import org.apache.streampark.common.util.ClassLoaderUtils;
2930
import org.apache.streampark.common.util.DeflaterUtils;
3031
import org.apache.streampark.common.util.ExceptionUtils;
3132
import org.apache.streampark.common.util.FlinkConfigurationUtils;
@@ -39,6 +40,7 @@
3940
import org.apache.streampark.flink.client.bean.SubmitRequest;
4041
import org.apache.streampark.flink.client.bean.SubmitResponse;
4142
import org.apache.streampark.flink.client.bean.TriggerSavepointRequest;
43+
import org.apache.streampark.flink.client.conf.FlinkSavepointOptions;
4244
import org.apache.streampark.flink.core.FlinkClusterClient;
4345
import org.apache.streampark.flink.core.conf.FlinkRunOption;
4446

@@ -69,7 +71,6 @@
6971
import org.apache.flink.configuration.PipelineOptionsInternal;
7072
import org.apache.flink.python.PythonOptions;
7173
import org.apache.flink.runtime.jobgraph.JobGraph;
72-
import org.apache.flink.runtime.jobgraph.SavepointConfigOptions;
7374
import org.apache.flink.util.FlinkException;
7475
import org.apache.flink.util.Preconditions;
7576

@@ -221,6 +222,27 @@ protected void logSavepointClientRequest(String operation, SavepointRequestTrait
221222
logInfo(message.toString());
222223
}
223224

225+
/**
226+
* Submits a job graph, tolerating the signature change {@code ClusterClient#submitJob} went
227+
* through: it took a {@code JobGraph} until Flink 2.x widened the parameter to {@code
228+
* ExecutionPlan}, which {@code JobGraph} implements. The instance is accepted by either
229+
* version — only the declared parameter type moved — but this module is compiled once against
230+
* a single baseline, so a direct call binds to one signature and fails against the other with
231+
* {@code NoSuchMethodError}.
232+
*/
233+
private static String submitJobGraph(ClusterClient<?> client, JobGraph jobGraph) throws Exception {
234+
for (java.lang.reflect.Method method : client.getClass().getMethods()) {
235+
if ("submitJob".equals(method.getName())
236+
&& method.getParameterCount() == 1
237+
&& method.getParameterTypes()[0].isInstance(jobGraph)) {
238+
Object future = method.invoke(client, jobGraph);
239+
return ((java.util.concurrent.CompletableFuture<?>) future).get().toString();
240+
}
241+
}
242+
throw new FlinkException(
243+
"No ClusterClient#submitJob(..) accepting a JobGraph on " + client.getClass().getName());
244+
}
245+
224246
protected SubmitResponse submitJobGraphToCluster(
225247
SubmitRequest submitRequest,
226248
Configuration flinkConfig,
@@ -235,7 +257,7 @@ protected SubmitResponse submitJobGraphToCluster(
235257
PackagedProgram packageProgram = programJobGraph._1();
236258
JobGraph jobGraph = programJobGraph._2();
237259
ClusterClient<?> client = clientSupplier.call();
238-
String jobId = client.submitJob(jobGraph).get().toString();
260+
String jobId = submitJobGraph(client, jobGraph);
239261
SubmitResponse result =
240262
new SubmitResponse(
241263
clusterIdSupplier.call(),
@@ -385,7 +407,7 @@ private void applyPyFlinkConfig(SubmitRequest submitRequest, Configuration flink
385407
private void applyCommonPipelineConfig(SubmitRequest submitRequest, Configuration flinkConfig) {
386408
safeSet(flinkConfig, PipelineOptions.NAME, submitRequest.effectiveAppName());
387409
safeSet(flinkConfig, DeploymentOptions.TARGET, submitRequest.deployMode().getName());
388-
safeSet(flinkConfig, SavepointConfigOptions.SAVEPOINT_PATH, submitRequest.savePoint());
410+
safeSet(flinkConfig, FlinkSavepointOptions.SAVEPOINT_PATH, submitRequest.savePoint());
389411
safeSet(
390412
flinkConfig,
391413
ApplicationConfiguration.APPLICATION_MAIN_CLASS,
@@ -417,10 +439,10 @@ private void applySavepointConfig(SubmitRequest submitRequest, Configuration fli
417439
}
418440
safeSet(
419441
flinkConfig,
420-
SavepointConfigOptions.SAVEPOINT_PATH,
442+
FlinkSavepointOptions.SAVEPOINT_PATH,
421443
submitRequest.savePoint());
422-
flinkConfig.setBoolean(
423-
SavepointConfigOptions.SAVEPOINT_IGNORE_UNCLAIMED_STATE,
444+
flinkConfig.set(
445+
FlinkSavepointOptions.SAVEPOINT_IGNORE_UNCLAIMED_STATE,
424446
submitRequest.allowNonRestoredState());
425447
boolean enableRestoreMode =
426448
submitRequest.restoreMode() != null
@@ -508,6 +530,21 @@ protected SubmitResponse trySubmit(
508530
}
509531
}
510532

533+
/**
534+
* Makes the program's classloader resolve {@code org.apache.streampark.*} from its parent — the
535+
* shims classloader built for the registered Flink version — instead of from the submitted jar.
536+
*
537+
* <p>Set through the raw key rather than {@code CoreOptions}: the typed accessors around it
538+
* moved between Flink 1.x and 2.x, while the key itself did not, and this module is compiled
539+
* against a single baseline but submits to whichever version the user registered.
540+
*/
541+
private static Configuration streamParkParentFirstConfig() {
542+
Configuration configuration = new Configuration();
543+
configuration.setString(
544+
"classloader.parent-first-patterns.additional", "org.apache.streampark.");
545+
return configuration;
546+
}
547+
511548
public Tuple2<PackagedProgram, JobGraph> getJobGraph(
512549
Configuration flinkConfig, SubmitRequest submitRequest,
513550
File jarFile) throws Exception {
@@ -533,8 +570,20 @@ public Tuple2<PackagedProgram, JobGraph> getJobGraph(
533570
}
534571
} else {
535572
builder.setJarFile(jarFile);
536-
// BUG: https://github.com/apache/streampark/issues/3761
537-
// .setUserClassPaths(Lists.newArrayList(submitRequest.classPaths()))
573+
if (submitRequest.jobType() == FlinkJobType.FLINK_SQL) {
574+
// The FLINK_SQL fat jar bundles only the SQL client and the shims it was built
575+
// against; it carries none of the target Flink version's own jars, so those have to
576+
// be handed to the program explicitly. Scoped to FLINK_SQL, unlike the blanket
577+
// disable from https://github.com/apache/streampark/issues/3761, which was never
578+
// verified against this job type.
579+
builder.setUserClassPaths(Lists.newArrayList(submitRequest.classPaths()));
580+
// ...and the StreamPark classes must come from the parent — this thread runs under
581+
// the shims classloader for the *registered* Flink version, whereas the fat jar
582+
// carries whichever shims it happened to be built with. Loading both ends in a
583+
// LinkageError as soon as one references the other, and silently mixes Flink
584+
// versions when it does not.
585+
builder.setConfiguration(streamParkParentFirstConfig());
586+
}
538587
}
539588

540589
PackagedProgram packageProgram = builder.build();
@@ -587,7 +636,17 @@ <T> T getOptionFromDefaultFlinkConfig(String flinkHome, ConfigOption<T> option)
587636
List<CustomCommandLine> getCustomCommandLines(String flinkHome) {
588637
Configuration flinkDefaultConfiguration = getFlinkDefaultConfiguration(flinkHome);
589638
String confDir = flinkHome + "/conf";
590-
return CliFrontend.loadCustomCommandLines(flinkDefaultConfiguration, confDir);
639+
// CliFrontend/GenericCLI are bound to the Flink version bundled with this module (loaded by
640+
// this class's own classloader), but the calling thread's context classloader may currently
641+
// be a target-version shims classloader (see FlinkShimsProxy). GenericCLI's internal
642+
// ServiceLoader.load(PipelineExecutorFactory.class) resolves providers via the context
643+
// classloader, so leaving it as the shims classloader here would load a PipelineExecutorFactory
644+
// implementation from a different Flink version than the interface bundled here, throwing
645+
// ServiceConfigurationError ("not a subtype"). Force it back to this class's own classloader
646+
// for the duration of this call.
647+
return ClassLoaderUtils.runAsClassLoader(
648+
FlinkClientTrait.class.getClassLoader(),
649+
() -> CliFrontend.loadCustomCommandLines(flinkDefaultConfiguration, confDir));
591650
}
592651

593652
public Integer getParallelism(SubmitRequest submitRequest) {
@@ -596,8 +655,7 @@ public Integer getParallelism(SubmitRequest submitRequest) {
596655
submitRequest.getProp(ConfigKeys.KEY_FLINK_PARALLELISM()).toString());
597656
}
598657
return getFlinkDefaultConfiguration(submitRequest.flinkVersion().getFlinkHome())
599-
.getInteger(
600-
CoreOptions.DEFAULT_PARALLELISM, CoreOptions.DEFAULT_PARALLELISM.defaultValue());
658+
.get(CoreOptions.DEFAULT_PARALLELISM, CoreOptions.DEFAULT_PARALLELISM.defaultValue());
601659
}
602660

603661
Options getCommandLineOptions(String flinkHome) {

streampark-flink/streampark-flink-packer/src/main/java/org/apache/streampark/flink/packer/pipeline/AbstractFlinkBuildResponse.java

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,11 +40,13 @@ protected AbstractFlinkBuildResponse(String workspacePath, boolean pass) {
4040
}
4141

4242
@Override
43+
@JsonProperty("workspacePath")
4344
public String workspacePath() {
4445
return workspacePath;
4546
}
4647

4748
@Override
49+
@JsonProperty("pass")
4850
public boolean pass() {
4951
return pass;
5052
}

streampark-flink/streampark-flink-packer/src/main/java/org/apache/streampark/flink/packer/pipeline/DockerImageBuildResponse.java

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,14 +52,17 @@ public DockerImageBuildResponse(
5252
this.dockerInnerMainJarPath = dockerInnerMainJarPath;
5353
}
5454

55+
@JsonProperty("flinkImageTag")
5556
public String flinkImageTag() {
5657
return flinkImageTag;
5758
}
5859

60+
@JsonProperty("podTemplatePaths")
5961
public Map<String, String> podTemplatePaths() {
6062
return podTemplatePaths;
6163
}
6264

65+
@JsonProperty("dockerInnerMainJarPath")
6366
public String dockerInnerMainJarPath() {
6467
return dockerInnerMainJarPath;
6568
}

0 commit comments

Comments
 (0)