Discussion:
How to give lesser memory(-xmx) for logwriter process and higher for worker process?
(too old to reply)
saurabh mimani
2018-11-10 03:41:02 UTC
Permalink
Version Info:
"org.apache.storm" % "storm-core" % "1.2.1"
"org.apache.storm" % "storm-kafka-client" % "1.2.1"

With this version of storm, when running topology, I see two processes for
each worker: one for logwriter and other for worker, as mentioned here
<https://stackoverflow.com/a/42517250/1610034> as well.

The problem is I want to give less memory to logwriter process(say
-Xmx512mb) and higher to worker process(say -Xmx12gb). But I am not able to
achieve this, I have tried playing with different config.

There is config topology.worker.logwriter.childopts, which is taken by
logwriter process, but than I have given some config to worker.childopts ,
but that also gets applied to both logwriter and worker process. Similary I
tried using topology.worker.childopts and that also gets applied to both
logwriter and worker process.

How to solve this to give lesser memory for logwriter process and higher
for worker process?

Best Regards

Saurabh Kumar Mimani
Stig Rohde Døssing
2018-11-10 15:25:34 UTC
Permalink
Are you sure topology.worker.childopts are actually being applied to the
logwriter JVM? Keep in mind that the entire worker launch command is being
passed through as command line parameters to the logwriter main method, so
topology.worker.childopts will appear in the command line for the
logwriter, but the parameters should appear after the
"org.apache.storm.LogWriter"
part, so shouldn't be applied to the logwriter JVM.

See
https://github.com/apache/storm/blob/eaed3cbf3610bbefb9466f2c40db1e0e2a20ca9c/storm-server/src/main/java/org/apache/storm/daemon/supervisor/BasicContainer.java#L643
(this is for 2.0.0, but I'd imagine the same thing happens in 1.x)

Den lÞr. 10. nov. 2018 kl. 04.48 skrev saurabh mimani <
Post by saurabh mimani
"org.apache.storm" % "storm-core" % "1.2.1"
"org.apache.storm" % "storm-kafka-client" % "1.2.1"
With this version of storm, when running topology, I see two processes for
each worker: one for logwriter and other for worker, as mentioned here
<https://stackoverflow.com/a/42517250/1610034> as well.
The problem is I want to give less memory to logwriter process(say
-Xmx512mb) and higher to worker process(say -Xmx12gb). But I am not able to
achieve this, I have tried playing with different config.
There is config topology.worker.logwriter.childopts, which is taken by
logwriter process, but than I have given some config to worker.childopts ,
but that also gets applied to both logwriter and worker process. Similary I
tried using topology.worker.childopts and that also gets applied to both
logwriter and worker process.
How to solve this to give lesser memory for logwriter process and higher
for worker process?
Best Regards
Saurabh Kumar Mimani
Loading...