Zookeeper : increase tick time

[Lina FAHED]

Hello,

i’m new in Apache Storm, i have a problem that the Zookeeper tick time is very small for the treatment i need, so the session is terminated early.

I’m using a pre-built version, so how could i increase the tick time for Zookeeper, as it’s not possible to set it in the storm.yaml ?

Please, i need help. Thanks in advance.

Regards.

[Matthias J. Sax]

Hi,
this is a Zookeeper setting (and not a Storm parameter). You need to
update ZK config. For example, /opt/zookeeper/conf/zoo.cfg

-Matthias

Post by Lina FAHED
Hello,
i’m new in Apache Storm, i have a problem that the Zookeeper tick time is very small for the treatment i need, so the session is terminated early.
I’m using a pre-built version, so how could i increase the tick time for Zookeeper, as it’s not possible to set it in the storm.yaml ?
Please, i need help. Thanks in advance.
Regards.

[Matthias J. Sax]

Hi,
this is a Zookeeper setting (and not a Storm parameter). You need to
update ZK config. For example, /opt/zookeeper/conf/zoo.cfg

-Matthias

Post by Lina FAHED
Hello,
i’m new in Apache Storm, i have a problem that the Zookeeper tick time is very small for the treatment i need, so the session is terminated early.
I’m using a pre-built version, so how could i increase the tick time for Zookeeper, as it’s not possible to set it in the storm.yaml ?
Please, i need help. Thanks in advance.
Regards.

[Lina FAHED]

Hi,

as the Zookeeper is embedded in the Storm version, so, i didn’t found the zookeeper in /opt/
so, how to access it ?

Thanks,

Lina

Post by Matthias J. Sax
Hi,
this is a Zookeeper setting (and not a Storm parameter). You need to
update ZK config. For example, /opt/zookeeper/conf/zoo.cfg
-Matthias

Post by Lina FAHED
Hello,
i’m new in Apache Storm, i have a problem that the Zookeeper tick time is very small for the treatment i need, so the session is terminated early.
I’m using a pre-built version, so how could i increase the tick time for Zookeeper, as it’s not possible to set it in the storm.yaml ?
Please, i need help. Thanks in advance.
Regards.

[Matthias J. Sax]

What do you mean by "embedded in the Storm version"? Zookeeper must be
installed on your cluster manually:
https://storm.apache.org/documentation/Setting-up-a-Storm-cluster.html

Depending on your ZK installation directory, the config file might be in
a different directory of course.

-Matthias

Hi,
as the Zookeeper is embedded in the Storm version, so, i didn’t found the zookeeper in /opt/
so, how to access it ?
Thanks,
Lina

Post by Matthias J. Sax
Hi,
this is a Zookeeper setting (and not a Storm parameter). You need to
update ZK config. For example, /opt/zookeeper/conf/zoo.cfg
-Matthias

Post by Lina FAHED
Hello,
i’m new in Apache Storm, i have a problem that the Zookeeper tick time is very small for the treatment i need, so the session is terminated early.
I’m using a pre-built version, so how could i increase the tick time for Zookeeper, as it’s not possible to set it in the storm.yaml ?
Please, i need help. Thanks in advance.
Regards.

[Lina FAHED]

i’m using a local mode of Storm, so the Zookeeper in embedded in the Storm release i installed,
i didn’t found a path for ZK in order to change its configurations.

do you think that it would be better to turn on a cluster mode ?

thanks

Lina

Post by Matthias J. Sax
What do you mean by "embedded in the Storm version"? Zookeeper must be
https://storm.apache.org/documentation/Setting-up-a-Storm-cluster.html
Depending on your ZK installation directory, the config file might be in
a different directory of course.
-Matthias

Post by Lina FAHED
Hi,
as the Zookeeper is embedded in the Storm version, so, i didn’t found the zookeeper in /opt/
so, how to access it ?
Thanks,
Lina

Post by Matthias J. Sax
Hi,
this is a Zookeeper setting (and not a Storm parameter). You need to
update ZK config. For example, /opt/zookeeper/conf/zoo.cfg
-Matthias

Post by Lina FAHED
Hello,
i’m new in Apache Storm, i have a problem that the Zookeeper tick time is very small for the treatment i need, so the session is terminated early.
I’m using a pre-built version, so how could i increase the tick time for Zookeeper, as it’s not possible to set it in the storm.yaml ?
Please, i need help. Thanks in advance.
Regards.

[Matthias J. Sax]

You could try to increase the session and/or connection timeout within
Storm:

Config.STORM_ZOOKEEPER_CONNECTION_TIMEOUT
Config.STORM_ZOOKEEPER_SESSION_TIMEOUT

-Matthias

i’m using a local mode of Storm, so the Zookeeper in embedded in the Storm release i installed,
i didn’t found a path for ZK in order to change its configurations.
do you think that it would be better to turn on a cluster mode ?
thanks
Lina

Post by Matthias J. Sax
What do you mean by "embedded in the Storm version"? Zookeeper must be
https://storm.apache.org/documentation/Setting-up-a-Storm-cluster.html
Depending on your ZK installation directory, the config file might be in
a different directory of course.
-Matthias

Hi,
as the Zookeeper is embedded in the Storm version, so, i didn’t found the zookeeper in /opt/
so, how to access it ?
Thanks,
Lina

Post by Matthias J. Sax
Hi,
this is a Zookeeper setting (and not a Storm parameter). You need to
update ZK config. For example, /opt/zookeeper/conf/zoo.cfg
-Matthias

Post by Lina FAHED
Hello,
i’m new in Apache Storm, i have a problem that the Zookeeper tick time is very small for the treatment i need, so the session is terminated early.
I’m using a pre-built version, so how could i increase the tick time for Zookeeper, as it’s not possible to set it in the storm.yaml ?
Please, i need help. Thanks in advance.
Regards.

[Lina FAHED]

i tried previously to set the ZK connection and the session time out to more than 30mn (via storm.yaml). But within less than 1mn, i have the following exception

45431 [main-SendThread(localhost:2000)] INFO org.apache.storm.zookeeper.ClientCnxn - Client session timed out, have not heard from server in 27264ms for sessionid 0x14f6956c7ba0003, closing socket connection and attempting reconnect
45438 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] WARN org.apache.storm.zookeeper.server.NIOServerCnxn - caught end of stream exception
org.apache.storm.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read additional data from client sessionid 0x14f6956c7ba0005, likely client has closed socket

i noticed (in the log) that the ZK session timeout cannot be changed : i have the following :
org.apache.storm.zookeeper.server.ZooKeeperServer - Created server with tickTime 2000 minSessionTimeout 4000 maxSessionTimeout 40000

so maybe the Client session is timed out because of the ZK « small » tickTime. I have a doubt that the changes i made are not really considered by storm.

Thanks,

Lina

Post by Matthias J. Sax
You could try to increase the session and/or connection timeout within
Config.STORM_ZOOKEEPER_CONNECTION_TIMEOUT
Config.STORM_ZOOKEEPER_SESSION_TIMEOUT
-Matthias

i’m using a local mode of Storm, so the Zookeeper in embedded in the Storm release i installed,
i didn’t found a path for ZK in order to change its configurations.
do you think that it would be better to turn on a cluster mode ?
thanks
Lina

Post by Matthias J. Sax
What do you mean by "embedded in the Storm version"? Zookeeper must be
https://storm.apache.org/documentation/Setting-up-a-Storm-cluster.html
Depending on your ZK installation directory, the config file might be in
a different directory of course.
-Matthias

Hi,
as the Zookeeper is embedded in the Storm version, so, i didn’t found the zookeeper in /opt/
so, how to access it ?
Thanks,
Lina

Post by Matthias J. Sax
Hi,
this is a Zookeeper setting (and not a Storm parameter). You need to
update ZK config. For example, /opt/zookeeper/conf/zoo.cfg
-Matthias

Post by Lina FAHED
Hello,
i’m new in Apache Storm, i have a problem that the Zookeeper tick time is very small for the treatment i need, so the session is terminated early.
I’m using a pre-built version, so how could i increase the tick time for Zookeeper, as it’s not possible to set it in the storm.yaml ?
Please, i need help. Thanks in advance.
Regards.

[Matthias J. Sax]

Might be an issue in LocalCluster. I have no other ideas. You might also
ask at the dev-list... Or setup a "real" cluster on you local machine.

Sorry that I can't help further.

-Matthias

Post by Lina FAHED
i tried previously to set the ZK connection and the session time out to
more than 30mn (via storm.yaml). But within less than 1mn, i have the
following exception
45431 [main-SendThread(localhost:2000)] INFO
org.apache.storm.zookeeper.ClientCnxn - *Client session timed out*, have
not heard from server in 27264ms for sessionid 0x14f6956c7ba0003,
closing socket connection and attempting reconnect
45438 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] WARN
org.apache.storm.zookeeper.server.NIOServerCnxn - caught end of stream
exception
Unable to read additional data from client sessionid 0x14f6956c7ba0005,
likely client has closed socket
i noticed (in the log) that the ZK session timeout cannot be changed : i
org.apache.storm.zookeeper.server.ZooKeeperServer - Created server with
*tickTime 2000* minSessionTimeout 4000 maxSessionTimeout 40000
so maybe the Client session is timed out because of the ZK « small »
tickTime. I have a doubt that the changes i made are not really
considered by storm.
Thanks,
Lina

Le 27 août 2015 à 10:53, Matthias J. Sax
You could try to increase the session and/or connection timeout within
Config.STORM_ZOOKEEPER_CONNECTION_TIMEOUT
Config.STORM_ZOOKEEPER_SESSION_TIMEOUT
-Matthias

i’m using a local mode of Storm, so the Zookeeper in embedded in the
Storm release i installed,
i didn’t found a path for ZK in order to change its configurations.
do you think that it would be better to turn on a cluster mode ?
thanks
Lina

Le 27 août 2015 à 10:31, Matthias J. Sax
What do you mean by "embedded in the Storm version"? Zookeeper must be
https://storm.apache.org/documentation/Setting-up-a-Storm-cluster.html
Depending on your ZK installation directory, the config file might be in
a different directory of course.
-Matthias

Hi,
as the Zookeeper is embedded in the Storm version, so, i didn’t
found the zookeeper in /opt/
so, how to access it ?
Thanks,
Lina

Le 27 août 2015 à 10:19, Matthias J. Sax
Hi,
this is a Zookeeper setting (and not a Storm parameter). You need to
update ZK config. For example, /opt/zookeeper/conf/zoo.cfg
-Matthias

Post by Lina FAHED
Hello,
i’m new in Apache Storm, i have a problem that the Zookeeper tick
time is very small for the treatment i need, so the session is
terminated early.
I’m using a pre-built version, so how could i increase the tick
time for Zookeeper, as it’s not possible to set it in the
storm.yaml ?
Please, i need help. Thanks in advance.
Regards.